Sandia’s first machine learning and deep learning conference

Last year, a computer program beat a professional Go player for the first time.  Go is even more difficult for computers than games such as chess. The program was trained by playing two computerized Go systems against each other, using deep learning, a kind of machine learning. Each system combined two artificial neural networks, one to learn the next best move and another to predict the winner of the game. Of course, machine learning and deep learning algorithms can be applied to significant real world challenges, including the Labs’ specialty in national security.

Deep learning uses multiple layers of artificial neural networks to learn complex patterns in data, such as recognizing and classifying images and recognizing and parsing language. The term “deep” comes from having multiple layers of biologically inspired networks, which was extremely taxing until recently, says Tim Draelos (6362), an artificial neural network expert and co-founder of Sandia’s Machine Learning and Deep Learning Conference along with Danny Dunlavy (1461), a machine learning expert.

Systems Mission Engineering Center 6300 and the Data Science Research Challenge sponsored the two-day conference held this August. The goal was to bring together researchers, practitioners, and interns from across the Labs working in the areas of machine learning and deep learning. Presenters came from 13 Centers and the six technical Divisions. The conference was held at the Steve Schiff Auditorium and the 904 Auditorium at Sandia/California.

“The types of missions we support generate massive amounts of data and overwhelm the ability of the human in the loop to process the information and make critical decisions in real time,” says Eunice Young (6360), a sponsor of the conference. “What if software could learn to recognize complex patterns and then make inferences and decisions along with the human in the loop? Our choice is to wait for industry to provide an answer, or join the forefront of research in this area. This conference pulls together the diversity of thought at the Labs and focuses attention on what we do best: solve difficult problems.”

“Our choice is to wait for industry to provide an answer, or join the forefront of research in this area.”

One of Tim’s research projects takes speech-recognition-like deep learning algorithms and applies them to challenges in the seismic signal and event detection arena. Determining the difference between a natural earthquake and unnatural underground explosions requires event discrimination and identifying the phase of the seismic waves. “I think those two have a very good chance at being successful,” says Tim, because they’re similar to the pattern recognition challenges in speech recognition.

Shane Alvarez (6362-1), an intern working with Tim, presented on his work adapting a speech recognition artificial neural network to telling the difference between an earthquake and an underground explosion. Two of the challenges he faced this summer were the limited number of events in his training data and the fact that many seismic events are faint.

These challenges of limited and poor-quality training data are common to many of the applications for which Sandia is using machine learning and deep learning. Several of the talks presented possible remedies for these, and other common challenges.

Systems engineer Matt Kagie (6791) spoke on an evolution-inspired method to get around human bias and explore a wider variety of solutions in classifying or clustering data. The example he used was analyzing satellite data to classify different kinds of lightning. The unsupervised algorithm included several randomizing methods to try to find the best approach and evaluated itself on a number of criteria before feeding the best solutions back into the evolution stage.

Jacob Caswell (5852), who recently transitioned from a year-round intern to a software engineering staff position, found the conference to be quite helpful. He says, “This is a nice opportunity to see how machine learning and deep learning might pertain to my new position and see how it’s being applied all over the Labs. I also got a chance to meet and talk with the people who are actually using machine learning and deep learning.”

Tim says he hopes the machine learning and deep learning conference will become an annual event. “The commercial deep learning community has spent a lot money, a lot of energy, and has invested in a lot of smart people in developing these algorithms for problems that are close enough to challenges Sandia has,” says Tim. “I don’t think deep learning has really penetrated all the places in Sandia that it needs to yet.”