AI gears up for data analysis: making the most of machine learning

Large-scale experimental facilities such as neutron and synchrotron sources have become an essential element of modern scientific research, allowing visiting researchers to probe the structure and properties of many different types of materials. They also generate huge amounts of experimental data, which can make it difficult for visiting scientists without specialist knowledge of the experiment to extract meaningful information from the raw datasets. As a result, some of the data collected during their valuable beamtime is never properly analysed.

The good news is that this situation has improved dramatically over the last 10 years, with a consortium of leading neutron facilities working together to streamline and standardize the software used to analyse data from neutron scattering and muon spectroscopy experiments. The framework – called MANTiD – supports a common data structure and shared algorithms to enable visiting scientists to easily process and visualize their experimental results.

the next major challenge is to make it easier for researchers from different scientific backgrounds to analyse and interpret the complex experimental output that can be produced.

Today, the buzz surrounding artificial intelligence is hard to ignore. We’ve been wowed by computers that can beat grandmasters at chess and Go, and are served by increasingly powerful speech recognition and machine translation tools. To the list of highlights, you can also add breakthroughs in image recognition together with progress in driverless vehicles. But why is it all happening now? After all, many machine learning algorithms have been around for decades.

The crucial factor is the impact of scale, specifically the parallel growth of data and available computing power. And this has transformed the capabilities of one technique in particular – deep learning – which benefits greatly from the availability of large datasets.

While other methods plateau when you feed them with more information, the performance of deep learning’s artificial neural networks keeps climbing. And the larger (or deeper) the neural network, the greater its capacity to absorb the value of its inputs and deliver meaningful outputs.

Combining big data with large amounts of compute makes it possible to create artificial neural networks with many so-called hidden layers. These deep-learning systems are giant mathematical functions that comprise multiple layers of nodes, equipped with self-adjusting weights and biases, all sandwiched between a series of inputs and outputs.

The rich combination of data and compute – together with a greater understanding of how to train (or propagate) these powerful multi-layered networks – is now taking the performance of machine-learning techniques to new heights.

The flip-side is that research groups need access to large amounts of data and large amounts of compute to engage the full benefits of deep learning, and they need support from teams who can get these systems up and running.

Success in deploying AI requires teams with talent across multiple areas: an understanding of the data, knowledge of machine learning algorithms plus statistical methods, and expertise in high-performance or cluster computing. But the potential rewards make the challenges worth conquering and can extend to other areas beyond analysing experimental results.

Google has reportedly saved a fortune by using deep learning to reduce the costs of running its data centres. Algorithms can alert operators when machinery is close to failure and should be replaced, which minimizes downtime. The output can also inform optimal servicing frequencies to keep equipment in reliable working order for as long as possible.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s