$2.5 million grant to ‘pursue new directions’ in neural networks
Sandia researcher Eric C. Cyr has received a 2018 Early Career Research Program award of $500,000 every year for five years to improve deep neural networks so they more efficiently combine experimental results with the most complex computer models.
The national award from the DOE’s Office of Science is meant to “identify and provide support to those researchers early in their careers who have the potential to develop new scientific ideas, promote them and convince their peers to pursue them as new directions.”
Eric’s winning proposal was “Parallel-in-layer methods for Extreme-Scale Machine Learning.” Machine learning in the form of deep neural networks has seen unprecedented success in the past few years, leading to innovations in self-driving cars, pattern recognition (including facial and speech recognition) and natural language processing.
A deep neural network is a model whose architecture is loosely inspired by the brain. It
uses a sequence of mathematical operations collected in layers to automatically embed knowledge of previously observed data. Such models can make predictions and associations not previously observed; thus, machines “learn.”
Commercial innovations driving machine learning
Eric attributes recent successes of these mainly commercial innovations to the advent of high-density computing devices and to the large data sets used to train the computer-generated models. For his project, he proposes to advance the technology further into the realm of more complex scientific devices. In particular, he wrote in his proposal, his goal is to develop “an algorithmic toolset for scientific machine learning that will transform how experts integrate experimental data and computer simulations through the use of deep neural networks.”
Despite the success of deep neural networks, the cost of these approaches remains high, with training times measured in days on relatively small computer clusters, he said. Compared to the commercial applications of deep neural networks, scientific data sets are massive, measuring in terabytes to petabytes. To Eric, algorithmic advances are required to robustly apply machine learning technologies to these demanding data sets.
Another problem: approaches used in commercial deep-neural-network architectures don’t generalize well to scientific data sets.
These issues limit the applicability of deep neural networks as a general tool for use in scientific machine learning.
Eric aims to reduce or eliminate these limitations.
DOE supports lifetime discovery science
“Supporting talented researchers early in their career is key to building and maintaining a skilled and effective scientific workforce for the nation,” said Energy Secretary Rick Perry in a DOE news release. By investing in the next generation of scientific researchers, we are supporting lifelong discovery science to fuel the nation’s innovation system. We are proud of the accomplishments these young scientists have already made, and look forward to following their achievements in years to come.”
Eric was a summer 2002 intern at Sandia’s Computer Science Research Institute, before finishing his bachelor’s at Clemson University in 2003 and earning a doctorate in computer science from the University of Illinois at Urbana-Champaign in 2008. He joined Sandia in January 2009 as a postdoctoral researcher, became a senior member of the technical staff in 2010 and a principal member of the technical staff in 2015. He is a joint author on 25 published papers.
DOE Early Career grants are available in the program areas of advanced scientific computing research, biological and environmental research, basic energy sciences, fusion energy sciences, high energy physics and nuclear physics. To be eligible for the DOE award, a researcher must be an untenured, tenure-track assistant or associate professor at a U.S. academic institution or a full-time employee at a DOE national laboratory, who received a doctorate within the past 10 years.
Thirty researchers from DOE’s national laboratories and 54 from U.S. universities were selected for the prestigious award this year.