Modeling real-world phenomena to any degree of accuracy is a challenge that the scientific research community has navigated since its foundation. Lack of information and limited computational and observational resources necessitate modeling assumptions which, when invalid, lead to model-form error (MFE). The work reported herein explored a novel method to represent model-form uncertainty (MFU) that combines Bayesian statistics with the emerging field of universal differential equations (UDEs). The fundamental principle behind UDEs is simple: use known equational forms that govern a dynamical system when you have them; then incorporate data-driven approaches – in this case neural networks (NNs) – embedded within the governing equations to learn the interacting terms that were underrepresented. Utilizing epidemiology as our motivating exemplar, this report will highlight the challenges of modeling novel infectious diseases while introducing ways to incorporate NN approximations to MFE. Prior to embarking on a Bayesian calibration, we first explored methods to augment the standard (non-Bayesian) UDE training procedure to account for uncertainty and increase robustness of training. In addition, it is often the case that uncertainty in observations is significant; this may be due to randomness or lack of precision in the measurement process. This uncertainty typically manifests as “noisy” observations which deviate from a true underlying signal. To account for such variability, the NN approximation to MFE is endowed with a probabilistic representation and is updated using available observational data in a Bayesian framework. By representing the MFU explicitly and deploying an embedded, data-driven model, this approach enables an agile, expressive, and interpretable method for representing MFU. In this report we will provide evidence that Bayesian UDEs show promise as a novel framework for any science-based, data-driven MFU representation; while emphasizing that significant advances must be made in the calibration of Bayesian NNs to ensure a robust calibration procedure.
Prediction of flow, transport, and deformation in fractured and porous media is critical to improving our scientific understanding of coupled thermal-hydrological-mechanical processes related to subsurface energy storage and recovery, nonproliferation, and nuclear waste storage. Especially, earth rock response to changes in pressure and stress has remained a critically challenging task. In this work, we advance computational capabilities for coupled processes in fractured and porous media using Sandia Sierra Multiphysics software through verification and validation problems such as poro-elasticity, elasto-plasticity and thermo-poroelasticity. We apply Sierra software for geologic carbon storage, fluid injection/extraction, and enhanced geothermal systems. We also significantly improve machine learning approaches through latent space and self-supervised learning. Additionally, we develop new experimental technique for evaluating dynamics of compacted soils at an intermediate scale. Overall, this project will enable us to systematically measure and control the earth system response to changes in stress and pressure due to subsurface energy activities.
The focus of this project is to accelerate and transform the workflow of multiscale materials modeling by developing an integrated toolchain seamlessly combining DFT, SNAP, LAMMPS, (shown in Figure 1-1) and a machine-learning (ML) model that will more efficiently extract information from a smaller set of first-principles calculations. Our ML model enables us to accelerate first-principles data generation by interpolating existing high fidelity data, and extend the simulation scale by extrapolating high fidelity data (102 atoms) to the mesoscale (104 atoms). It encodes the underlying physics of atomic interactions on the microscopic scale by adapting a variety of ML techniques such as deep neural networks (DNNs), and graph neural networks (GNNs). We developed a new surrogate model for density functional theory using deep neural networks. The developed ML surrogate is demonstrated in a workflow to generate accurate band energies, total energies, and density of the 298K and 933K Aluminum systems. Furthermore, the models can be used to predict the quantities of interest for systems with more number of atoms than the training data set. We have demonstrated that the ML model can be used to compute the quantities of interest for systems with 100,000 Al atoms. When compared with 2000 Al system the new surrogate model is as accurate as DFT, but three orders of magnitude faster. We also explored optimal experimental design techniques to choose the training data and novel Graph Neural Networks to train on smaller data sets. These are promising methods that need to be explored in the future.
Tracer gases, whether they are chemical or isotopic in nature, are useful tools in examining the flow and transport of gaseous or volatile species in the underground. One application is using detection of short-lived argon and xenon radionuclides to monitor for underground nuclear explosions. However, even chemically inert species, such as the noble gases, have bene observed to exhibit non-conservative behavior when flowing through porous media containing certain materials, such as zeolites, due to gas adsorption processes. This report details the model developed, implemented, and tested in the open source and massively parallel subsurface flow and transport simulator PFLOTRAN for future use in modeling the transport of adsorbing tracer gases.
Ambient infrasound noise in quiet, rural environments has been extensively studied and well-characterized through noise models for several decades. More recently, creating noise models for high-noise rural environments has also become an area of active research. However, far less work has been done to create generalized low-frequency noise models for urban areas. The high ambient noise levels expected in cities and other highly populated areas means that these environments are regarded as poor locations for acoustic sensors, and historically, sensor deployment in urban areas were avoided for this reason. However, there are several advantages to placing sensors in urban environments, including convenience of deployment and maintenance, and increasingly, necessity, as more previously rural areas become populated. This study seeks to characterize trends in low-frequency urban noise by creating a background noise model for Las Vegas, NV, using the Las Vegas Infrasound Array (LVIA): a network of eleven infrasound sensors deployed throughout the city. Data included in this study spans from 2019 to 2021 and provides a largely uninterrupted record of noise levels in the city from 0.1–500 Hz, with only minor discontinuities on individual stations. We organize raw data from the LVIA sensors into hourly power spectral density (PSD) averages for each station and select from these PSDs to create frequency distributions for time periods of interest . These frequency distributions are converted into probability density functions (PDFs), which are then used to evaluate variations in frequency and amplitude over daily to seasonal timescale s. In addition to PDFs, the median, 5th percentile, and 95th percentile amplitude values are calculated across the entire frequency range. This methodology follows a well-established process for noise model creation.
This project explores the idea of performing kinetic numerical simulations in the Z inner magnetically insulated transmission line (inner MITL) by reduced physics models such as a guiding center drift kinetic approximation for particles and electrostatic and magnetostatic approximation for the fields. The basic problem explored herein is the generation, formation, and evolution of vortices by electron space charge limited (SCL) emission. The results indicate that for relevant to Z values of peak current and pulse length, these approximations are excellent, while also providing tens to hundreds of times reduction in the computational load. The benefits could be enormous: Implementation of these reduced physics models in present particle-in-cell (PIC) codes could enable them to be routinely used for experimental design while still capturing essential non-thermal (kinetic) physics.
Sandia National Laboratories (SNL) is designing and developing an Artificial Intelligence (AI)-enabled smart digital assistant (SDA), Inspecta (International Nuclear Safeguards Personal Examination and Containment Tracking Assistant). The goal is to provide inspectors an in-field digital assistant that can perform tasks identified as tedious, challenging, or prone to human error. During 2021, we defined the requirements for Inspecta based on reviews of International Atomic Energy Agency (IAEA) publications and interviews with former IAEA inspectors. We then mapped the requirements to current commercial or open-source technical capabilities to provide a development path for an initial Inspecta prototype while highlighting potential research and development tasks. We selected a highimpact inspection task that could be performed by an early Inspecta prototype and are developing the initial architecture, including hardware platform. This paper describes the methodology for selecting an initial task scenario, the first set of Inspecta skills needed to assist with that task scenario and finally the design and development of Inspecta’s architecture and platform.
Representation of soil organic carbon (SOC) dynamics in Earth system models (ESMs) is a key source of uncertainty in predicting carbon climate feedbacks. The magnitude of this uncertainty can be reduced by accurate representation of environmental controllers of SOC stocks in ESMs. In this study, we used data of environmental factors, field SOC observations, ESM projections and machine learning approaches to identify dominant environmental controllers of SOC stocks and derive functional relationships between environmental factors and SOC stocks. Our derived functional relationships predicted SOC stocks with similar accuracy as the machine learning approach. We used the derived relationships to benchmark the coupled model intercomparison project phase six ESM representation of SOC stocks. We found divergent environmental control representation in ESMs in comparison to field observations. Representation of SOC in ESMs can be improved by including additional environmental factors and representing their functional relationships with SOC consistent with observations.
This SAND report documents CIS Late Start LDRD Project 22-0311, "Differential geometric approaches to momentum-based formulations for fluids". The project primarily developed geometric mechanics formulations for momentum-based descriptions of nonrelativistic fluids, utilizing a differential geometry/exterior calculus treatment of momentum and a space+time splitting. Specifically, the full suite of geometric mechanics formulations (variational/Lagrangian, Lie-Poisson Hamiltonian and Curl-Form Hamiltonian) were developed in terms of exterior calculus using vector-bundle valued differential forms. This was done for a fairly general version of semi-direct product theory sufficient to cover a wide range of both neutral and charged fluid models, including compressible Euler, magnetohydrodynamics and Euler-Maxwell. As a secondary goal, this project also explored the connection between geometric mechanics formulations and the more traditional Godunov form (a hyperbolic system of conservation laws). Unfortunately, this stage did not produce anything particularly interesting, due to unforeseen technical difficulties. There are two publications related to this work currently in preparation, and this work will be presented at SIAM CSE 23, at which the PI is organizing a mini-symposium on geometric mechanics formulations and structure-preserving discretizations for fluids. The logical next step is to utilize the exterior calculus based understanding of momentum coupled with geometric mechanics formulations to develop (novel) structure-preserving discretizations of momentum. This is the main subject of a successful FY23 CIS LDRD "Structure-preserving discretizations for momentum-based formulations of fluids".
Estimation of two-phase fluid flow properties is important to understand and predict water and gas movement through the vadose zone for agricultural, hydrogeological, and engineering applications, such as for vapor-phase contaminant transport and/or containment of noble gases in the subsurface. In this second progress report of FY22, we present two ongoing activities related to imbibition testing on volcanic rock samples. We present the development of a new analytical solution predicting the temperature response observed during imbibition into dry samples, as discussed in our previous first progress report for FY22. We also illustrate the use of a multi-modal capillary pressure distribution to simulate both early- and late-time imbibition data collected on tuff core that can exhibit multiple pore types. These FY22 imbibition tests were conducted for an extended period (i.e., far beyond the time required for the wetting front to reach the top of the sample), which is necessary for parameter estimation and characterization of two different pore types within the samples.
Making reliable predictions in the presence of uncertainty is critical to high-consequence modeling and simulation activities, such as those encountered at Sandia National Laboratories. Surrogate or reduced-order models are often used to mitigate the expense of performing quality uncertainty analyses with high-fidelity, physics-based codes. However, phenomenological surrogate models do not always adhere to important physics and system properties. This project develops surrogate models that integrate physical theory with experimental data through a maximally-informative framework that accounts for the many uncertainties present in computational modeling problems. Correlations between relevant outputs are preserved through the use of multi-output or co-predictive surrogate models; known physical properties (specifically monotoncity) are also preserved; and unknown physics and phenomena are detected using a causal analysis. By endowing surrogate models with key properties of the physical system being studied, their predictive power is arguably enhanced, allowing for reliable simulations and analyses at a reduced computational cost.
Researchers have recently estimated that Arctic submarine permafrost currently traps 60 billion tons of methane and contains 560 billion tons of organic carbon in seafloor sediments and soil, a giant pool of carbon with potentially large feedbacks on the climate system. Unlike terrestrial permafrost, the submarine permafrost system has remained a “known unknown” because of the difficulty in acquiring samples and measurements. Consequently, this potentially large carbon stock never yet considered in global climate models or policy discussions, represents a real wildcard in our understanding of Earth’s climate. This report summarizes our group’s effort at developing a numerical modeling framework designed to produce a first-of-its-kind estimate of Arctic methane gas releases from the marine sediments to the water column, and potentially to the atmosphere, where positive climate feedback may occur. Newly developed modeling capability supported by the Laboratory Directed Research and Development (LDRD) program at Sandia National Laboratories now gives us the ability to probabilistically map gas distribution and quantity in the seabed by using a hybrid approach of geospatial machine learning, and predictive numerical thermodynamic ensemble modeling. The novelty in this approach is its ability to produce maps of useful data in regions that are only sparsely sampled, a common challenge in the Arctic, and a major obstacle to progress in the past. By applying this model to the circum-Arctic continental shelves and integrating the flux of free gas from in situ methanogenesis and dissociating gas hydrates from the sediment column under climate forcing, we can provide the most reliable estimate of a spatially and temporally varying source term for greenhouse gas flux that can be used by global oceanographic circulation and Earth system models (such as DOE’s E3SM). The result will allow us to finally tackle the wildcard of the submarine permafrost carbon system, and better inform us about the severity of future national security threats that sustained climate change poses.
Imaging using THz waves has been a promising option for penetrative measurements in environments that are opaque to visible wavelengths. However, available THz imaging systems have been limited to relatively low frame rates and cannot be applied to study fast dynamics. This work explores the use of upconversion imaging techniques based on nonlinear optics to enable wavelength-flexible high frame rate THz imaging. UpConversion Imaging (UCI) uses nonlinear conversion techniques to shift the THz wavelengths carrying a target image to shorter visible or near-IR wavelengths that can be detected by available high-speed cameras. This report describes the analysis methodology used to design a prototype high-rate THz UCI system and gives a detailed explanations of the design choices that were made. The design uses a high-rate pulse-burst laser system to pump both THz generation and THz upconversion detection, allowing for scaling to acquisition rates in excess of 10 kHz. The design of the prototype system described in this report has been completed and all necessary materials have been procured. Assembly and characterization testing is on-going at the submission of this report. This report proposes future directions for work on high-rate THz UCI and potential applications of future systems.
Simulating molecules and atomic systems at quantum accuracy is a grand challenge for science in the 21st century. Quantum-accurate simulations would enable the design of new medicines and the discovery of new materials. The defining problem in this challenge is that quantum calculations on large molecules, like proteins or DNA, are fundamentally impossible with current algorithms. In this work, we explore a range of different methods that aim to make large, quantum-accurate simulations possible. We show that using advanced classical models, we can accurately simulate ion channels, an important biomolecular system. We show how advanced classical models can be implemented in an exascale-ready software package. Lastly, we show how machine learning can learn the laws of quantum mechanics from data and enable quantum electronic structure calculations on thousands of atoms, a feat that is impossible for current algorithms. Altogether, this work shows that combining advances in physics models, computing, and machine learning, we are moving closer to the reality of accurately simulating our molecular world.
This report details work that was completed to address the Fiscal Year 2022 Advanced Science and Technology (AS&T) Laboratory Directed Research and Development (LDRD) call for “AI-enhanced Co-Design of Next Generation Microelectronics.” This project required concurrent contributions from the fields of 1) materials science, 2) devices and circuits, 3) physics of computing, and 4) algorithms and system architectures. During this project, we developed AI-enhanced circuit design methods that relied on reinforcement learning and evolutionary algorithms. The AI-enhanced design methods were tested on neuromorphic circuit design problems that have real-world applications related to Sandia’s mission needs. The developed methods enable the design of circuits, including circuits that are built from emerging devices, and they were also extended to enable novel device discovery. We expect that these AI-enhanced design methods will accelerate progress towards developing next-generation, high-performance neuromorphic computing systems.
This project was broadly motivated by the need for new hardware that can process information such as images and sounds right at the point of where the information is sensed (e.g. edge computing). The project was further motivated by recent discoveries by group demonstrating that while certain organic polymer blends can be used to fabricate elements of such hardware, the need to mix ionic and electronic conducting phases imposed limits on performance, dimensional scalability and the degree of fundamental understanding of how such devices operated. As an alternative to blended polymers containing distinct ionic and electronic conducting phases, in this LDRD project we have discovered that a family of mixed valence coordination compounds called Prussian blue analogue (PBAs), with an open framework structure and ability to conduct both ionic and electronic charge, can be used for inkjet-printed flexible artificial synapses that reversibly switch conductance by more than four orders of magnitude based on electrochemically tunable oxidation state. Retention of programmed states is improved by nearly two orders of magnitude compared to the extensively studied organic polymers, thus enabling in-memory compute and avoiding energy costly off-chip access during training. We demonstrate dopamine detection using PBA synapses and biocompatibility with living neurons, evoking prospective application for brain - computer interfacing. By application of electron transfer theory to in-situ spectroscopic probing of intervalence charge transfer, we elucidate a switching mechanism whereby the degree of mixed valency between N-coordinated Ru sites controls the carrier concentration and mobility, as supported by density functional theory (DFT) .
In this work we infer the underlying distribution on pore radius in human cortical bone samples using ultrasonic attenuation data. We first discuss how to formulate polydisperse attenuation models using a probabilistic approach and the Waterman Truell model for scattering attenuation. We then compare the Independent Scattering Approximation and the higher-order Waterman Truell models’ forward predictions for total attenuation in polydisperse samples. Following this, we formulate an inverse problem under the Prohorov Metric Framework coupled with variational regularization to stabilize this inverse problem. We then use experimental attenuation data taken from human cadaver samples and solve inverse problems resulting in nonparametric estimates of the probability density function on pore radius. We compare these estimates to the “true” microstructure of the bone samples determined via microCT imaging. We find that our methodology allows us to reliably estimate the underlying microstructure of the bone from attenuation data.
Structural modularity is critical to solid-state transformer (SST) and solid-state power substation (SSPS) concepts, but operational aspects related to this modularity are not yet fully understood. Previous studies and demonstrations of modular power conversion systems assume identical module compositions, but dependence on module uniformity undercuts the value of the modular framework. In this project, a hierarchical control approach was developed for modular SSTs which achieves system-level objectives while ensuring equitable power sharing between nonuniform building block modules. This enables module replacements and upgrades which leverage circuit and device technology advancements to improve system-level performance. The functionality of the control approach is demonstrated in detailed time-domain simulations. Results of this project provide context and strategic direction for future LDRD projects focusing on technologies supporting the SST crosscut outcome of the resilient energy systems mission campaign.
This project evaluated the use of emerging spintronic memory devices for robust and efficient variational inference schemes. Variational inference (VI) schemes, which constrain the distribution for each weight to be a Gaussian distribution with a mean and standard deviation, are a tractable method for calculating posterior distributions of weights in a Bayesian neural network such that this neural network can also be trained using the powerful backpropagation algorithm. Our project focuses on domain-wall magnetic tunnel junctions (DW-MTJs), a powerful multi-functional spintronic synapse design that can achieve low power switching while also opening the pathway towards repeatable, analog operation using fabricated notches. Our initial efforts to employ DW-MTJs as an all-in-one stochastic synapse with both a mean and standard deviation didn’t end up meeting the quality metrics for hardware-friendly VI. In the future, new device stacks and methods for expressive anisotropy modification may make this idea still possible. However, as a fall back that immediately satisfies our requirements, we invented and detailed how the combination of a DW-MTJ synapse encoding the mean and a probabilistic Bayes-MTJ device, programmed via a ferroelectric or ionically modifiable layer, can robustly and expressively implement VI. This design includes a physics-informed small circuit model, that was scaled up to perform and demonstrate rigorous uncertainty quantification applications, up to and including small convolutional networks on a grayscale image classification task, and larger (Residual) networks implementing multi-channel image classification. Lastly, as these results and ideas all depend upon the idea of an inference application where weights (spintronic memory states) remain non-volatile, the retention of these synapses for the notched case was further interrogated. These investigations revealed and emphasized the importance of both notch geometry and anisotropy modification in order to further enhance the endurance of written spintronic states. In the near future, these results will be mapped to effective predictions for room temperature and elevated operation DW-MTJ memory retention, and experimentally verified when devices become available.
This report is a summary of a 3-year LDRD project that developed novel methods to detect faults in the electric power grid dramatically faster than today’s protection systems. Accurately detecting and quickly removing electrical faults is imperative for power system resilience and national security to minimize impacts to defense critical infrastructure. The new protection schemes will improve grid stability during disturbances and allow additional integration of renewable energy technologies with low inertia and low fault currents. Signal-based fast tripping schemes were developed that use the physics of the grid and do not rely on communication to reduce cyber risks for safely removing faults.
This report documents the progress made in simulating the HERMES-III Magnetically Insulated Transmission Line (MITL) and courtyard with EMPIRE and ITS. This study focuses on the shots that were taken during the months of June and July of 2019 performed with the new MITL extension. There were a few shots where there was dose mapping of the courtyard, 11132, 11133, 11134, 11135, 11136, and 11146. This report focuses on these shots because there was full data return from the MITL electrical diagnostics and the radiation dose sensors in the courtyard. The comparison starts with improving the processing of the incoming voltage into the EMPIRE simulation from the experiment. The currents are then compared at several location along the MITL. The simulation results of the electrons impacting the anode are shown. The electron impact energy and angle is then handed off to ITS which calculates the dose on the faceplate and locations in the courtyard and they are compared to experimental measurements. ITS also calculates the photons and electrons that are injected into the courtyard, these quantities are then used by EMPIRE to calculated the photon and electron transport in the courtyard. The details for the algorithms used to perform the courtyard simulations are presented as well as qualitative comparisons of the electric field, magnetic field, and the conductivity in the courtyard. Because of the computational burden of these calculations the pressure was reduce in the courtyard to reduce the computational load. The computation performance is presented along with suggestion on how to improve both the computational performance as well as the algorithmic performance. Some of the algorithmic changed would reduce the accuracy of the models and detail comparison of these changes are left for a future study. As well as, list of code improvements there is also a list of suggested experimental improvements to improve the quality of the data return.
The plant polymer lignin is the most abundant renewable source of aromatics on the planet and conversion of it to valuable fuels and chemicals is critical to the economic viability of a lignocellulosic biofuels industry and to meeting the DOE’s 2022 goal of $\$2.50$/gallon mean biofuel selling price. Presently, there is no efficient way of converting lignin into valuable commodities. Current biological approaches require mixtures of expensive ligninolytic enzymes and engineered microbes. This project was aimed at circumventing these problems by discovering commensal relationships among fungi and bacteria involved in biological lignin utilization and using this knowledge to engineer microbial communities capable of converting lignin into renewable fuels and chemicals. Essentially, we aimed to learn from, mimic and improve on nature. We discovered fungi that synergistically work together to degrade lignin, engineered fungal systems to increase expression of the required enzymes and engineered organisms to produce products such as biodegradable plastics precursors.
The MelMACCS User Guide is intended to assist analysts in how to use the MelMACCS application to create an interface file that can be used in a MACCS calculation. MelMACCS combines MELCOR results, in the form of a MELCOR plot file, with user input. This information is used to create a MelMACCS output file that is in a format compatible with MACCS. It can then be imported as an input file in WinMACCS and used for MACCS calculations.