Cyberattacks against industrial control systems have increased over the last decade, making it more critical than ever for system owners to have the tools necessary to understand the cyber resilience of their systems. However, existing tools are often qualitative, subject matter expertise-driven, or highly generic, making thorough, data-driven cyber resilience analysis challenging. The ADROC project proposed to develop a platform to enable efficient, repeatable, data-driven cyber resilience analysis for cyber-physical systems. The approach consists of two phases of modeling: computationally efficient math modeling and high-fidelity emulations. The first phase allows for scenarios of low concern to be quickly filtered out, conserving resources available for analysis. The second phase supports more detailed scenario analysis, which is more predictive of real-world systems. Data extracted from experiments is used to calculate cyber resilience metrics. ADROC then ranks scenarios based on these metrics, enabling prioritization of system resources to improve cyber resilience.
This report summarizes the activities performed as part of the Science and Engineering of Cybersecurity by Uncertainty quantification and Rigorous Experimentation (SECURE) Grand Challenge LDRD project. We provide an overview of the research done in this project, including work on cyber emulation, uncertainty quantification, and optimization. We present examples of integrated analyses performed on two case studies: a network scanning/detection study and a malware command and control study. We highlight the importance of experimental workflows and list references of papers and presentations developed under this project. We outline lessons learned and suggestions for future work.
Cyber testbeds provide an important mechanism for experimentally evaluating cyber security performance. However, as an experimental discipline, reproducible cyber experimentation is essential to assure valid, unbiased results. Even minor differences in setup, configuration, and testbed components can have an impact on the experiments, and thus, reproducibility of results. This paper documents a case study in reproducing an earlier emulation study, with the reproduced emulation experiment conducted by a different research group on a different testbed. We describe lessons learned as a result of this process, both in terms of the reproducibility of the original study and in terms of the different testbed technologies used by both groups. This paper also addresses the question of how to compare results between two groups' experiments, identifying candidate metrics for comparison and quantifying the results in this reproduction study.
Space systems provide many critical functions to the military, federal agencies, and infrastructure networks. Nation-state adversaries have shown the ability to disrupt critical infrastructure through cyber-attacks targeting systems of networked, embedded computers. Moving target defenses (MTDs) have been proposed as a means for defending various networks and systems against potential cyber-attacks. MTDs differ from many cyber resilience technologies in that they do not necessarily require detection of an attack to mitigate the threat. We devised a MTD algorithm and tested its application to a real-time network. We demonstrated MTD usage with a real-time protocol given constraints not typically found in best-effort networks. Second, we quantified the cyber resilience benefit of MTD given an exfiltration attack by an adversary. For our experiment, we employed MTD which resulted in a reduction of adversarial knowledge by 97%. Even when the adversary can detect when the address changes, there is still a reduction in adversarial knowledge when compared to static addressing schemes. Furthermore, we analyzed the core performance of the algorithm and characterized its unpredictability using nine different statistical metrics. The characterization highlighted the algorithm has good unpredictability characteristics with some opportunity for improvement to produce more randomness.
To combat dynamic, cyber-physical disturbances in the electric grid, online and adaptive remedial action schemes (RASs) are needed to achieve fast and effective response. However, a major challenge lies in reducing the computational burden of analyses needed to inform selection of appropriate controls. This paper proposes the use of a role and interaction discovery (RID) algorithm that leverages control sensitivities to gain insight into the controller roles and support groups. Using these results, a procedure is developed to reduce the control search space to reduce computation time while achieving effective control response. A case study is presented that considers corrective line switching to mitigate geomagnetically induced current (GIC) -saturated reactive power losses in a 20-bus test system. Results demonstrated both significant reduction of both the control search space and reactive power losses using the RID approach.
Aging plants, efficiency goals, and safety needs are driving increased digitalization in nuclear power plants (NPP). Security has always been a key design consideration for NPP architectures, but increased digitalization and the emergence of malware such as Stuxnet, CRASHOVERRIDE, and TRITON that specifically target industrial control systems have heightened concerns about the susceptibility of NPPs to cyber attacks. The cyber security community has come to realize the impossibility of guaranteeing the security of these plants with 100% certainty, so demand for including resilience in NPP architectures is increasing. Whereas cyber security design features often focus on preventing access by cyber threats and ensuring confidentiality, integrity, and availability (CIA) of control systems, cyber resilience design features complement security features by limiting damage, enabling continued operations, and facilitating a rapid recovery from the attack in the event control systems are compromised. This paper introduces the REsilience VeRification UNit (RevRun) toolset, a software platform that was prototyped to support cyber resilience analysis of NPP architectures. Researchers at Sandia National Laboratories have recently developed models of NPP control and SCADA systems using the SCEPTRE platform. SCEPTRE integrates simulation, virtual hardware, software, and actual hardware to model the operation of cyber-physical systems. RevRun can be used to extract data from SCEPTRE experiments and to process that data to produce quantitative resilience metrics of the NPP architecture modeled in SCEPTRE. This paper details how RevRun calculates these metrics in a customizable, repeatable, and automated fashion that limits the burden placed upon the analyst. This paper describes RevRun's application and use in the context of a hypothetical attack on an NPP control system. The use case specifies the control system and a series of attacks and explores the resilience of the system to the attacks. The use case further shows how to configure RevRun to run experiments, how resilience metrics are calculated, and how the resilience metrics and RevRun tool can be used to conduct the related resilience analysis.
Trust in a microelectronics-based system can be characterized as the level of confidence that a system is free of subversive alterations made during system development, or that the development process of a system has not been manipulated by a malicious adversary. Trust in systems has become an increasing concern over the past decade. This article presents a novel game-theoretic framework, called GPLADD (Graph-based Probabilistic Learning Attacker and Dynamic Defender), for analyzing and quantifying system trustworthiness at the end of the development process, through the analysis of risk of development-time system manipulation. GPLADD represents attacks and attacker-defender contests over time. It treats time as an explicit constraint and allows incorporating the informational asymmetries between the attacker and defender into analysis. GPLADD includes an explicit representation of attack steps via multi-step attack graphs, attacker and defender strategies, and player actions at different times. GPLADD allows quantifying the attack success probability over time and the attacker and defender costs based on their capabilities and strategies. This ability to quantify different attacks provides an input for evaluation of trust in the development process. We demonstrate GPLADD on an example attack and its variants. We develop a method for representing success probability for arbitrary attacks and derive an explicit analytic characterization of success probability for a specific attack. We present a numeric Monte Carlo study of a small set of attacks, quantify attack success probabilities, attacker and defender costs, and illustrate the options the defender has for limiting the attack success and improving trust in the development process.
Energy resilience has emerged as a national security priority over the past fifteen years. Recent research efforts have aimed to develop metrics and analysis methods for energy resilience, but most of those efforts have focused on extreme weather and other natural hazards as the threat of interest. This paper introduces a novel set of resilience metrics and exemplifies how they can be applied to analyze resilience for growing concerns about cyber threats. The metrics are formally described with mathematical equations and demonstrated in a case study that evaluates the resilience benefits of a new moving target defense technology.
Control systems for critical infrastructure are becoming increasingly interconnected while cyber threats against critical infrastructure are becoming more sophisticated and difficult to defend against. Historically, cyber security has emphasized building defenses to prevent loss of confidentiality, integrity, and availability in digital information and systems, but in recent years cyber attacks have demonstrated that no system is impenetrable and that control system operation may be detrimentally impacted. Cyber resilience has emerged as a complementary priority that seeks to ensure that digital systems can maintain essential performance levels, even while capabilities are degraded by a cyber attack. This paper examines how cyber security and cyber resilience may be measured and quantified in a control system environment. Load Frequency Control is used as an illustrative example to demonstrate how cyber attacks may be represented within mathematical models of control systems, to demonstrate how these events may be quantitatively measured in terms of cyber security or cyber resilience, and the differences and similarities between the two mindsets. These results demonstrate how various metrics are applied, the extent of their usability, and how it is important to analyze cyber-physical systems in a comprehensive manner that accounts for all the various parts of the system.
Considerable effort is currently being spent designing neuromorphic hardware for addressing challenging problems in a variety of pattern-matching applications. These neuromorphic systems offer low power architectures with intrinsically parallel and simple spiking neuron processing elements. Unfortunately, these new hardware architectures have been largely developed without a clear justification for using spiking neurons to compute quantities for problems of interest. Specifically, the use of spiking for encoding information in time has not been explored theoretically with complexity analysis to examine the operating conditions under which neuromorphic computing provides a computational advantage (time, space, power, etc.) In this paper, we present and formally analyze the use of temporal coding in a neural-inspired algorithm for optimization-based computation in neural spiking architectures.
Grid resilience is a concept related to a power system's ability to continue operating and delivering power even in the event that low probability, high-consequence disruptions such as hurricanes, earthquakes, and cyber-attacks occur. Grid resilience objectives focus on managing and, ideally, minimizing potential consequences that occur as a result of these disruptions. Currently, no formal grid resilience definitions, metrics, or analysis methods have been universally accepted. This document describes an effort to develop and describe grid resilience metrics and analysis methods. The metrics and methods described herein extend upon the Resilience Analysis Process (RAP) developed by Watson et al. for the 2015 Quadrennial Energy Review. The extension allows for both outputs from system models and for historical data to serve as the basis for creating grid resilience metrics and informing grid resilience planning and response decision-making. This document describes the grid resilience metrics and analysis methods. Demonstration of the metrics and methods is shown through a set of illustrative use cases.
Improved validation for models of complex systems has been a primary focus over the past year for the Resilience in Complex Systems Research Challenge. This document describes a set of research directions that are the result of distilling those ideas into three categories of research -- epistemic uncertainty, strong tests, and value of information. The content of this document can be used to transmit valuable information to future research activities, update the Resilience in Complex Systems Research Challenge's roadmap, inform the upcoming FY18 Laboratory Directed Research and Development (LDRD) call and research proposals, and facilitate collaborations between Sandia and external organizations. The recommended research directions can provide topics for collaborative research, development of proposals, workshops, and other opportunities.