Cyberattacks against industrial control systems have increased over the last decade, making it more critical than ever for system owners to have the tools necessary to understand the cyber resilience of their systems. However, existing tools are often qualitative, subject matter expertise-driven, or highly generic, making thorough, data-driven cyber resilience analysis challenging. The ADROC project proposed to develop a platform to enable efficient, repeatable, data-driven cyber resilience analysis for cyber-physical systems. The approach consists of two phases of modeling: computationally efficient math modeling and high-fidelity emulations. The first phase allows for scenarios of low concern to be quickly filtered out, conserving resources available for analysis. The second phase supports more detailed scenario analysis, which is more predictive of real-world systems. Data extracted from experiments is used to calculate cyber resilience metrics. ADROC then ranks scenarios based on these metrics, enabling prioritization of system resources to improve cyber resilience.
This report summarizes the activities performed as part of the Science and Engineering of Cybersecurity by Uncertainty quantification and Rigorous Experimentation (SECURE) Grand Challenge LDRD project. We provide an overview of the research done in this project, including work on cyber emulation, uncertainty quantification, and optimization. We present examples of integrated analyses performed on two case studies: a network scanning/detection study and a malware command and control study. We highlight the importance of experimental workflows and list references of papers and presentations developed under this project. We outline lessons learned and suggestions for future work.
Cyber testbeds provide an important mechanism for experimentally evaluating cyber security performance. However, as an experimental discipline, reproducible cyber experimentation is essential to assure valid, unbiased results. Even minor differences in setup, configuration, and testbed components can have an impact on the experiments, and thus, reproducibility of results. This paper documents a case study in reproducing an earlier emulation study, with the reproduced emulation experiment conducted by a different research group on a different testbed. We describe lessons learned as a result of this process, both in terms of the reproducibility of the original study and in terms of the different testbed technologies used by both groups. This paper also addresses the question of how to compare results between two groups' experiments, identifying candidate metrics for comparison and quantifying the results in this reproduction study.
Space systems provide many critical functions to the military, federal agencies, and infrastructure networks. Nation-state adversaries have shown the ability to disrupt critical infrastructure through cyber-attacks targeting systems of networked, embedded computers. Moving target defenses (MTDs) have been proposed as a means for defending various networks and systems against potential cyber-attacks. MTDs differ from many cyber resilience technologies in that they do not necessarily require detection of an attack to mitigate the threat. We devised a MTD algorithm and tested its application to a real-time network. We demonstrated MTD usage with a real-time protocol given constraints not typically found in best-effort networks. Second, we quantified the cyber resilience benefit of MTD given an exfiltration attack by an adversary. For our experiment, we employed MTD which resulted in a reduction of adversarial knowledge by 97%. Even when the adversary can detect when the address changes, there is still a reduction in adversarial knowledge when compared to static addressing schemes. Furthermore, we analyzed the core performance of the algorithm and characterized its unpredictability using nine different statistical metrics. The characterization highlighted the algorithm has good unpredictability characteristics with some opportunity for improvement to produce more randomness.
To combat dynamic, cyber-physical disturbances in the electric grid, online and adaptive remedial action schemes (RASs) are needed to achieve fast and effective response. However, a major challenge lies in reducing the computational burden of analyses needed to inform selection of appropriate controls. This paper proposes the use of a role and interaction discovery (RID) algorithm that leverages control sensitivities to gain insight into the controller roles and support groups. Using these results, a procedure is developed to reduce the control search space to reduce computation time while achieving effective control response. A case study is presented that considers corrective line switching to mitigate geomagnetically induced current (GIC) -saturated reactive power losses in a 20-bus test system. Results demonstrated both significant reduction of both the control search space and reactive power losses using the RID approach.
Aging plants, efficiency goals, and safety needs are driving increased digitalization in nuclear power plants (NPP). Security has always been a key design consideration for NPP architectures, but increased digitalization and the emergence of malware such as Stuxnet, CRASHOVERRIDE, and TRITON that specifically target industrial control systems have heightened concerns about the susceptibility of NPPs to cyber attacks. The cyber security community has come to realize the impossibility of guaranteeing the security of these plants with 100% certainty, so demand for including resilience in NPP architectures is increasing. Whereas cyber security design features often focus on preventing access by cyber threats and ensuring confidentiality, integrity, and availability (CIA) of control systems, cyber resilience design features complement security features by limiting damage, enabling continued operations, and facilitating a rapid recovery from the attack in the event control systems are compromised. This paper introduces the REsilience VeRification UNit (RevRun) toolset, a software platform that was prototyped to support cyber resilience analysis of NPP architectures. Researchers at Sandia National Laboratories have recently developed models of NPP control and SCADA systems using the SCEPTRE platform. SCEPTRE integrates simulation, virtual hardware, software, and actual hardware to model the operation of cyber-physical systems. RevRun can be used to extract data from SCEPTRE experiments and to process that data to produce quantitative resilience metrics of the NPP architecture modeled in SCEPTRE. This paper details how RevRun calculates these metrics in a customizable, repeatable, and automated fashion that limits the burden placed upon the analyst. This paper describes RevRun's application and use in the context of a hypothetical attack on an NPP control system. The use case specifies the control system and a series of attacks and explores the resilience of the system to the attacks. The use case further shows how to configure RevRun to run experiments, how resilience metrics are calculated, and how the resilience metrics and RevRun tool can be used to conduct the related resilience analysis.
Trust in a microelectronics-based system can be characterized as the level of confidence that a system is free of subversive alterations made during system development, or that the development process of a system has not been manipulated by a malicious adversary. Trust in systems has become an increasing concern over the past decade. This article presents a novel game-theoretic framework, called GPLADD (Graph-based Probabilistic Learning Attacker and Dynamic Defender), for analyzing and quantifying system trustworthiness at the end of the development process, through the analysis of risk of development-time system manipulation. GPLADD represents attacks and attacker-defender contests over time. It treats time as an explicit constraint and allows incorporating the informational asymmetries between the attacker and defender into analysis. GPLADD includes an explicit representation of attack steps via multi-step attack graphs, attacker and defender strategies, and player actions at different times. GPLADD allows quantifying the attack success probability over time and the attacker and defender costs based on their capabilities and strategies. This ability to quantify different attacks provides an input for evaluation of trust in the development process. We demonstrate GPLADD on an example attack and its variants. We develop a method for representing success probability for arbitrary attacks and derive an explicit analytic characterization of success probability for a specific attack. We present a numeric Monte Carlo study of a small set of attacks, quantify attack success probabilities, attacker and defender costs, and illustrate the options the defender has for limiting the attack success and improving trust in the development process.
Energy resilience has emerged as a national security priority over the past fifteen years. Recent research efforts have aimed to develop metrics and analysis methods for energy resilience, but most of those efforts have focused on extreme weather and other natural hazards as the threat of interest. This paper introduces a novel set of resilience metrics and exemplifies how they can be applied to analyze resilience for growing concerns about cyber threats. The metrics are formally described with mathematical equations and demonstrated in a case study that evaluates the resilience benefits of a new moving target defense technology.
Control systems for critical infrastructure are becoming increasingly interconnected while cyber threats against critical infrastructure are becoming more sophisticated and difficult to defend against. Historically, cyber security has emphasized building defenses to prevent loss of confidentiality, integrity, and availability in digital information and systems, but in recent years cyber attacks have demonstrated that no system is impenetrable and that control system operation may be detrimentally impacted. Cyber resilience has emerged as a complementary priority that seeks to ensure that digital systems can maintain essential performance levels, even while capabilities are degraded by a cyber attack. This paper examines how cyber security and cyber resilience may be measured and quantified in a control system environment. Load Frequency Control is used as an illustrative example to demonstrate how cyber attacks may be represented within mathematical models of control systems, to demonstrate how these events may be quantitatively measured in terms of cyber security or cyber resilience, and the differences and similarities between the two mindsets. These results demonstrate how various metrics are applied, the extent of their usability, and how it is important to analyze cyber-physical systems in a comprehensive manner that accounts for all the various parts of the system.
Considerable effort is currently being spent designing neuromorphic hardware for addressing challenging problems in a variety of pattern-matching applications. These neuromorphic systems offer low power architectures with intrinsically parallel and simple spiking neuron processing elements. Unfortunately, these new hardware architectures have been largely developed without a clear justification for using spiking neurons to compute quantities for problems of interest. Specifically, the use of spiking for encoding information in time has not been explored theoretically with complexity analysis to examine the operating conditions under which neuromorphic computing provides a computational advantage (time, space, power, etc.) In this paper, we present and formally analyze the use of temporal coding in a neural-inspired algorithm for optimization-based computation in neural spiking architectures.
Grid resilience is a concept related to a power system's ability to continue operating and delivering power even in the event that low probability, high-consequence disruptions such as hurricanes, earthquakes, and cyber-attacks occur. Grid resilience objectives focus on managing and, ideally, minimizing potential consequences that occur as a result of these disruptions. Currently, no formal grid resilience definitions, metrics, or analysis methods have been universally accepted. This document describes an effort to develop and describe grid resilience metrics and analysis methods. The metrics and methods described herein extend upon the Resilience Analysis Process (RAP) developed by Watson et al. for the 2015 Quadrennial Energy Review. The extension allows for both outputs from system models and for historical data to serve as the basis for creating grid resilience metrics and informing grid resilience planning and response decision-making. This document describes the grid resilience metrics and analysis methods. Demonstration of the metrics and methods is shown through a set of illustrative use cases.
Improved validation for models of complex systems has been a primary focus over the past year for the Resilience in Complex Systems Research Challenge. This document describes a set of research directions that are the result of distilling those ideas into three categories of research -- epistemic uncertainty, strong tests, and value of information. The content of this document can be used to transmit valuable information to future research activities, update the Resilience in Complex Systems Research Challenge's roadmap, inform the upcoming FY18 Laboratory Directed Research and Development (LDRD) call and research proposals, and facilitate collaborations between Sandia and external organizations. The recommended research directions can provide topics for collaborative research, development of proposals, workshops, and other opportunities.
This report contains the written footprint of a Sandia-hosted workshop held in Albuquerque, New Mexico, June 22-23, 2016 on “Complex Systems Models and Their Applications: Towards a New Science of Verification, Validation and Uncertainty Quantification,” as well as of pre-work that fed into the workshop. The workshop’s intent was to explore and begin articulating research opportunities at the intersection between two important Sandia communities: the complex systems (CS) modeling community, and the verification, validation and uncertainty quantification (VVUQ) community The overarching research opportunity (and challenge) that we ultimately hope to address is: how can we quantify the credibility of knowledge gained from complex systems models, knowledge that is often incomplete and interim, but will nonetheless be used, sometimes in real-time, by decision makers?
The Integrated Human Futures Project provides a set of analytical and quantitative modeling and simulation tools that help explore the links among human social, economic, and ecological conditions, human resilience, conflict, and peace, and allows users to simulate tradeoffs and consequences associated with different future development and mitigation scenarios. In the current study, we integrate five distinct modeling platforms to simulate the potential risk of social unrest in Egypt resulting from the Grand Ethiopian Renaissance Dam (GERD) on the Blue Nile in Ethiopia. The five platforms simulate hydrology, agriculture, economy, human ecology, and human psychology/behavior, and show how impacts derived from development initiatives in one sector (e.g., hydrology) might ripple through to affect other sectors and how development and security concerns may be triggered across the region. This approach evaluates potential consequences, intended and unintended, associated with strategic policy actions that span the development-security nexus at the national, regional, and international levels. Model results are not intended to provide explicit predictions, but rather to provide system-level insight for policy makers into the dynamics among these interacting sectors, and to demonstrate an approach to evaluating short- and long-term policy trade-offs across different policy domains and stakeholders. The GERD project is critical to government-planned development efforts in Ethiopia but is expected to reduce downstream freshwater availability in the Nile Basin, fueling fears of negative social and economic impacts that could threaten stability and security in Egypt. We tested these hypotheses and came to the following preliminary conclusions. First, the GERD will have an important short-term impact on water availability, food production, and hydropower production in Egypt, depending on the short- term reservoir fill rate. Second, the GERD will have a very small impact on water availability in the Nile Basin over the longer term. Depending on the GERD fill rate, short-term (e.g., within its first 5 years of operation) annual losses in Egyptian food production may peak briefly at 25 percent. Long-term (e.g., 15 to 30 year) cumulative losses in Egypt's food production may be less than 3 percent regardless of the fill rate, with the GERD having essentially no impact on projected annual food production in Egypt about 25 years after opening. For the quick fill rates, the short-term losses may be sufficient to create an important decrease in overall household health among the general population, which, along with other economic stressors and different strategies employed by the government, could lead to social unrest. Third, and perhaps most importantly, our modeling suggests that the GERD's effect on Egypt's food and water resources is small when compared to the effect of projected Egyptian population and economic growth (and the concomitant increase in water consumption). The latter dominating factors are exacerbated in the modeling by natural climate variability and may be further exacerbated by climate change. Our modeling suggests that these growth dynamics combine to create long-term water scarcity in Egypt, regardless of the Ethiopian project. All else being equal, filling strategies that employ slow fill rates for the GERD (e.g., 8 to 13 years) may mitigate the risks in future scenarios for Egypt somewhat, but no policy or action regarding the GERD is likely to significantly alleviate the projected water scarcity in Egypt's Nile Basin. However, general beliefs among the Egyptian populace regarding the GERD as a major contributing factor for scarcities in Egypt could make Ethiopia a scapegoat for Egyptian grievances -- contributing to social unrest in Egypt and generating undesirable (and unnecessary) tension between these two countries. Such tension could threaten the constructive relationships between Egypt and Ethiopia that are vital to maintaining stability and security within and between their respective regional spheres of influence, Middle East and North Africa, and the Horn of Africa.
Background Recent declines in US cigarette smoking prevalence have coincided with increases in use of other tobacco products. Multiple product tobacco models can help assess the population health impacts associated with use of a wide range of tobacco products. Methods and Findings We present a multi-state, dynamical systems population structure model that can be used to assess the effects of tobacco product use behaviors on population health. The model incorporates transition behaviors, such as initiation, cessation, switching, and dual use, related to the use of multiple products. The model tracks product use prevalence and mortality attributable to tobacco use for the overall population and by sex and age group. The model can also be used to estimate differences in these outcomes between scenarios by varying input parameter values. We demonstrate model capabilities by projecting future cigarette smoking prevalence and smoking-attributable mortality and then simulating the effects of introduction of a hypothetical new lower-risk tobacco product under a variety of assumptions about product use. Sensitivity analyses were conducted to examine the range of population impacts that could occur due to differences in input values for product use and risk. We demonstrate that potential benefits from cigarette smokers switching to the lower-risk product can be offset over time through increased initiation of this product. Model results show that population health benefits are particularly sensitive to product risks and initiation, switching, and dual use behaviors. Conclusion Our model incorporates the variety of tobacco use behaviors and risks that occur with multiple products. As such, it can evaluate the population health impacts associated with the introduction of new tobacco products or policies that may result in product switching or dual use. Further model development will include refinement of data inputs for non-cigarette tobacco products and inclusion of health outcomes such as morbidity and disability.
This report presents a mathematical model of the way in which a hospital uses a variety of resources, utilities and consumables to provide care to a set of in-patients, and how that hospital might adapt to provide treatment to a few patients with a serious infectious disease, like the Ebola virus. The intended purpose of the model is to support requirements planning studies, so that hospitals may be better prepared for situations that are likely to strain their available resources. The current model is a prototype designed to present the basic structural elements of a requirements planning analysis. Some simple illustrati ve experiments establish the mo del's general capabilities. With additional inve stment in model enhancement a nd calibration, this prototype could be developed into a useful planning tool for ho spital administrators and health care policy makers.
Adaptation is believed to be a source of resilience in systems. It has been difficult to measure the contribution of adaptation to resilience, unlike other resilience mechanisms such as restoration and recovery. One difficulty comes from treating adaptation as a deus ex machina that is interjected after a disruption. This provides no basis for bounding possible adaptive responses. We can bracket the possible effects of adaptation when we recognize that it occurs continuously, and is in part responsible for the current system’s properties. In this way the dynamics of the system’s pre-disruption structure provides information about post-disruption adaptive reaction. Seen as an ongoing process, adaptation has been argued to produce “robust-yet-fragile” systems. Such systems perform well under historical stresses but become committed to specific features of those stresses in a way that makes them vulnerable to system-level collapse when those features change. In effect adaptation lessens the cost of disruptions within a certain historical range, at the expense of increased cost from disruptions outside that range. Historical adaptive responses leave a signature in the structure of the system. Studies of ecological networks have suggested structural metrics that pick out systemic resilience in the underlying ecosystems. If these metrics are generally reliable indicators of resilience they provide another strategy for gaging adaptive resilience. To progress in understanding how the process of adaptation and the property of resilience interrelate in infrastructure systems, we pose some specific questions: Does adaptation confer resilience?; Does it confer resilience to novel shocks as well, or does it tune the system to fragility?; Can structural features predict resilience to novel shocks?; Are there policies or constraints on the adaptive process that improve resilience?.
The study develops a novel stochastic frontier modeling approach to the gravity equation for rare earth element (REE) trade between China and its trading partners between 2001 and 2009. The novelty lies in differentiating betweenbehind the border' trade costs by China and theimplicit beyond the border costs' of China's trading partners. Results indicate that the significance level of the independent variables change dramatically over the time period. While geographical distance matters for trade flows in both periods, the effect of income on trade flows is significantly attenuated, possibly capturing the negative effects of financial crises in the developed world. Second, the total export losses due tobehind the border' trade costs almost tripled over the time period. Finally, looking atimplicit beyond the border' trade costs, results show China gaining in some markets, although it is likely that some countries are substituting away from Chinese REE exports.
This report provides the results of a scoping study evaluating the potential risk reduction value of a hypothetical, earthquake early-warning system. The study was based on an analysis of the actions that could be taken to reduce risks to population and infrastructures, how much time would be required to take each action and the potential consequences of false alarms given the nature of the action. The results of the scoping analysis indicate that risks could be reduced through improving existing event notification systems and individual responses to the notification; and production and utilization of more detailed risk maps for local planning. Detailed maps and training programs, based on existing knowledge of geologic conditions and processes, would reduce uncertainty in the consequence portion of the risk analysis. Uncertainties in the timing, magnitude and location of earthquakes and the potential impacts of false alarms will present major challenges to the value of an early-warning system.
Sandia National Laboratories has developed several models to analyze potential consequences of homeland security incidents. Two of these models (the National Infrastructure Simulation and Analysis Center Agent-Based Laboratory for Economics, N-ABLE{trademark}, and Loki) simulate detailed facility- and product-level consequences of simulated disruptions to supply chains. Disruptions in supply chains are likely to reduce production of some commodities, which may reduce economic activity across many other types of supply chains throughout the national economy. The detailed nature of Sandia's models means that simulations are limited to specific supply chains in which detailed facility-level data has been collected, but policymakers are often concerned with the national-level economic impacts of supply-chain disruptions. A preliminary input-output methodology has been developed to estimate national-level economic impacts based upon the results of supply-chain-level simulations. This methodology overcomes two primary challenges. First, the methodology must be relatively simple to integrate successfully with existing models; it must be easily understood, easily applied to the supply-chain models without user intervention, and run quickly. The second challenge is more fundamental: the methodology must account for both upstream and downstream impacts that result from supply-chain disruptions. Input-output modeling typically estimates only upstream impacts, but shortages resulting from disruptions in many supply chains (for example, energy, communications, and chemicals) are likely to have large downstream impacts. In overcoming these challenges, the input-output methodology makes strong assumptions about technology and substitution. This paper concludes by applying the methodology to chemical supply chains.
Critical infrastructure resilience has become a national priority for the U. S. Department of Homeland Security. System resilience has been studied for several decades in many different disciplines, but no standards or unifying methods exist for critical infrastructure resilience analysis. This report documents the results of a late-start Laboratory Directed Research and Development (LDRD) project that investigated the identification of optimal recovery strategies that maximize resilience. To this goal, we formulate a bi-level optimization problem for infrastructure network models. In the 'inner' problem, we solve for network flows, and we use the 'outer' problem to identify the optimal recovery modes and sequences. We draw from the literature of multi-mode project scheduling problems to create an effective solution strategy for the resilience optimization model. We demonstrate the application of this approach to a set of network models, including a national railroad model and a supply chain for Army munitions production.
Critical infrastructure resilience has become a national priority for the U. S. Department of Homeland Security. System resilience has been studied for several decades in many different disciplines, but no standards or unifying methods exist for critical infrastructure resilience analysis. Few quantitative resilience methods exist, and those existing approaches tend to be rather simplistic and, hence, not capable of sufficiently assessing all aspects of critical infrastructure resilience. This report documents the results of a late-start Laboratory Directed Research and Development (LDRD) project that investigated the development of quantitative resilience through application of control design methods. Specifically, we conducted a survey of infrastructure models to assess what types of control design might be applicable for critical infrastructure resilience assessment. As a result of this survey, we developed a decision process that directs the resilience analyst to the control method that is most likely applicable to the system under consideration. Furthermore, we developed optimal control strategies for two sets of representative infrastructure systems to demonstrate how control methods could be used to assess the resilience of the systems to catastrophic disruptions. We present recommendations for future work to continue the development of quantitative resilience analysis methods.
Tensions between the energy and water sectors occur when demand for electric power is high and water supply levels are low. There are several regions of the country, such as the western and southwestern states, where the confluence of energy and water is always strained due to population growth. However, for much of the country, this tension occurs at particular times of year (e.g., summer) or when a region is suffering from drought conditions. This report discusses prior work on the interdependencies between energy and water. It identifies the types of power plants that are most likely to be susceptible to water shortages, the regions of the country where this is most likely to occur, and policy options that can be applied in both the energy and water sectors to address the issue. The policy options are designed to be applied in the near term, applicable to all areas of the country, and to ease the tension between the energy and water sectors by addressing peak power demand or decreased water supply.
Non-equilibrium sorption of contaminants in ground water systems is examined from the perspective of sorption rate estimation. A previously developed Markov transition probability model for solute transport is used in conjunction with a new conditional probability-based model of the sorption and desorption rates based on breakthrough curve data. Two models for prediction of spatially varying sorption and desorption rates along a one-dimensional streamline are developed. These models are a Markov model that utilizes conditional probabilities to determine the rates and an ensemble Kalman filter (EnKF) applied to the conditional probability method. Both approaches rely on a previously developed Markov-model of mass transfer, and both models assimilate the observed concentration data into the rate estimation at each observation time. Initial values of the rates are perturbed from the true values to form ensembles of rates and the ability of both estimation approaches to recover the true rates is examined over three different sets of perturbations. The models accurately estimate the rates when the mean of the perturbations are zero, the unbiased case. For the cases containing some bias, addition of the ensemble Kalman filter is shown to improve accuracy of the rate estimation by as much as an order of magnitude.
Proliferation resistance features that reduce the likelihood of diversion of nuclear materials from the civilian nuclear power fuel cycle are critical for a global nuclear future. A framework that monitors process information continuously can demonstrate the ability to resist proliferation by measuring and reducing diversion risk, thus ensuring the legitimate use of the nuclear fuel cycle. The automation of new nuclear facilities requiring minimal manual operation makes this possible by generating instantaneous system state data that can be used to track and measure the status of the process and material at any given time. Sandia National Laboratories (SNL) and the Japan Atomic Energy Agency (JAEA) are working in cooperation to develop an advanced transparency framework capable of assessing diversion risk in support of overall plant transparency. The ''diversion risk'' quantifies the probability and consequence of a host nation diverting nuclear materials from a civilian fuel cycle facility. This document introduces the details of the diversion risk quantification approach to be demonstrated in the fuel handling training model of the MONJU Fast Reactor.
Non-equilibrium sorption of contaminants in ground water systems is examined from the perspective of sorption rate estimation. A previously developed Markov transition probability model for solute transport is used in conjunction with a new conditional probability-based model of the sorption and desorption rates based on breakthrough curve data. Two models for prediction of spatially varying sorption and desorption rates along a one-dimensional streamline are developed. These models are a Markov model that utilizes conditional probabilities to determine the rates and an ensemble Kalman filter (EKF) applied to the conditional probability method. Both approaches rely on a previously developed Markov-model of mass transfer, and both models assimilate the observed concentration data into the rate estimation at each observation time. Initial values of the rates are perturbed from the true values to form ensembles of rates and the ability of both estimation approaches to recover the true rates is examined over three different sets of perturbations. The models accurately estimate the rates when the mean of the perturbations are zero, the unbiased case. Finally, for the cases containing some bias, addition of the ensemble Kalman filter is shown to improve accuracy of the rate estimation by as much as an order of magnitude.