Publications

Abstract not provided.

Abstract not provided.

Abstract not provided.

Malicious cyber-attacks are becoming increasingly prominent due to the advance of technology and methods over the last decade. These attacks have the potential to bring down critical infrastructures, such as nuclear power plants (NPP’s), which are so vital to the country that their incapacitation would have debilitating effects on national security, public health, or safety. Despite the devastating effects a cyber-attack could have on NPP’s, there is a lack of understanding as to the effects on the plant from a discreet failure or surreptitious sabotage of components and a lack of knowledge in how the control room operators would react to such a situation. In this project, the authors are collaborating with NPP operators to discern the impact of cyber-attacks on control room operations and lay out a framework to better understand the control room operators’ tasks and decision points.

Abstract not provided.

Abstract not provided.

Abstract not provided.

There are gaps in understanding how a cyber-attack would manifest itself within power plants and what these events would look like within the control room from an operator’s perspective. This is especially true for nuclear power plants where safety has much broader consequences than nonnuclear plants. The operating and emergency procedures that operators currently use are likely inadequate for targeted cyber-attacks. This research focuses on understanding how a cyber event would affect the operation of the plant, how an operator would perceive the event, and if the operator’s actions would keep the plant in a safe condition. This research is part of Sandia’s Laboratory Directed Research and Development program where a nuclear power plant cyber model of the control system digital architecture is coupled with a generic pressurized water reactor plant training simulator. Cyber event scenarios will be performed on the coupled system with plant operators. The scenarios simulate plant conditions that may exist during a cyber-attack, component failure, or insider sabotage, and provide an understanding of the displayed information and the actual plant conditions. These scenarios will determine if plant operators can 1) recognize that they are under cyber-attack and 2) take appropriate actions to keep the plant safe. This will also provide the opportunity to assess the operator cognitive workload during such events and identify where improvements might be made. Experiments with nuclear power plant operators will be carried out over FY 2018 and results of the research are expected by the end of FY 2018.

Abstract not provided.

Abstract not provided.

The charter for adversarial delay is to hinder access to critical resources through the use of physical systems increasing an adversarys task time. The traditional method for characterizing access delay has been a simple model focused on accumulating times required to complete each task with little regard to uncertainty, complexity, or decreased efficiency associated with multiple sequential tasks or stress. The delay associated with any given barrier or path is further discounted to worst-case, and often unrealistic, times based on a high-level adversary, resulting in a highly conservative calculation of total delay. This leads to delay systems that require significant funding and personnel resources in order to defend against the assumed threat, which for many sites and applications becomes cost prohibitive. A new methodology has been developed that considers the uncertainties inherent in the problem to develop a realistic timeline distribution for a given adversary path. This new methodology incorporates advanced Bayesian statistical theory and methodologies, taking into account small sample size, expert judgment, human factors and threat uncertainty. The result is an algorithm that can calculate a probability distribution function of delay times directly related to system risk. Through further analysis, the access delay analyst or end user can use the results in making informed decisions while weighing benefits against risks, ultimately resulting in greater system effectiveness with lower cost.

Over the past 2 decades, the U.S. nuclear power plant (NPP) fire protection community and overseas has been transitioning toward risk-informed and performance-based (RI/PB) practice in design, operation and regulation. To make more realistic decisions for risk-informed regulation, fire probabilistic risk analysis (PRA) methods needed further development. To address this need, in 2001, the U.S. Nuclear Regulatory Commission's (NRCs) Office of Nuclear Regulatory Research (RES) and the Electric Power Research Institute (EPRI) collaborated under a joint Memorandum of Understanding (MOU) to develop NUREG/CR-6850 (EPRI 101989), "EPRI/NRC-RES Fire PRA Methodology for Nuclear Power Facilities," a state-of-art fire PRA methodology. The fire human reliability analysis (HRA) guidance provided in NUREG/CR-6850 included: (1) a process for identification and inclusion of the human failure events (HFEs), (2) a methodology for assigning quantitative screening values to these HFEs, and (3) initial considerations of performance shaping factors (PSFs) and related fire effects that might need to be addressed in developing best-estimate human error probabilities (HEPs). However, NUREG/CR-6850 did not identify or produce a methodology to develop these best-estimate HEPs given the PSFs and the fire-related effects. In 2007, EPRI and RES embarked upon another cooperative project - building on existing HRA methods - to develop explicit guidance for estimating HEPs for human error events under fire-generated conditions. This collaborative project produced draft NUREG-1921, "EPRI/NRC-RES Fire Human Reliability Analysis Guidelines." The guidance presented in this report is intended to be both an improvement upon and an expansion of the initial guidance provided in NUREG/CR-6850. This paper will summarize the fire HRA guidance developed through this collaborative project, which addresses the range of fire procedures used in existing plants, the range of strategies for main control room (MCR) abandonment, and the potential impact of fire-induced electrical spurious actuation effects on crew performance. This guidance presents a three tiered, progressive approach for fire HRA quantification. The quantification approaches include: a screening approach per NUREG/CR-6850 guidance, a scoping approach, and detailed quantification using either EPRI's Cause-Based Decision Tree (CBDT) and Human cognitive Reliability/Operator Reliability Experiment (HCR/ORE) or NRC's A Technique for Human Event ANAlysis (ATHEANA) approach with modifications to account for fire effects. The newly developed scoping approach is intended to be less resource intensive than a detailed HRA, while providing less conservative HEPs than rough screening. The expectation is that the majority of the actions can be quantified using the scoping approach, thus detailed HRA will only be used for the more complex actions that do not meet the criteria for the scoping approach. It is anticipated that this guidance will be used by the industry as part of transition to the risk-informed, performance-based fire protection rule, 10 CFR 50.48c, that endorsed National Fire Protection Association (NFPA) 805, "Performance-Based Standard for Fire Protection for Light Water Reactor Electric Generating Plants" and possibly in response to other regulatory issues such as multiple spurious operation (MSO) and operator manual actions (OMAs). As the methodology is applied at a wide variety of NPPs, the guidance may benefit from future improvements to better support industry wide issues being addressed by fire PRAs.

This report summarizes research conducted through the Sandia National Laboratories Robust Automated Knowledge Capture Laboratory Directed Research and Development project. The objective of this project was to advance scientific understanding of the influence of individual cognitive attributes on decision making. The project has developed a quantitative model known as RumRunner that has proven effective in predicting the propensity of an individual to shift strategies on the basis of task and experience related parameters. Three separate studies are described which have validated the basic RumRunner model. This work provides a basis for better understanding human decision making in high consequent national security applications, and in particular, the individual characteristics that underlie adaptive thinking.

Abstract not provided.

Abstract not provided.

Abstract not provided.

Abstract not provided.

Abstract not provided.

Abstract not provided.

Human Reliability Analysis (HRA) methods have been developed primarily to provide information for use in probabilistic risk assessments analyzing nuclear power plant (NPP) operations. Despite this historical focus on the control room, there has been growing interest in applying HRA methods to other NPP activities such as dry cask storage operations (DCSOs) in which spent fuel is transferred into dry cask storage systems. This paper describes a successful application of aspects of the "A Technique for Human Event Analysis" (ATHEANA) HRA approach [1, 2] in performing qualitative HRA activities that generated insights on the potential for dropping a spent fuel cask during DCSOs. This paper provides a description of the process followed during the analysis, a description of the human failure event (HFE) scenario groupings, discussion of inferred human performance vulnerabilities, a detailed examination of one HFE scenario and illustrative approaches for avoiding or mitigating human performance vulnerabilities that may contribute to dropping a spent fuel cask.

Abstract not provided.

Abstract not provided.

Abstract not provided.

The Office of Nuclear Regulatory Research (RES) at the US Nuclear Regulatory Commission (USNRC) is sponsoring work in response to a Staff Requirements Memorandum (SRM) directing an effort to establish a single human reliability analysis (HRA) method for the agency or guidance for the use of multiple methods. As part of this effort an attempt to develop a comprehensive HRA qualitative approach is being pursued. This paper presents a draft of the method's middle layer, a part of the qualitative analysis phase that links failure mechanisms to performance shaping factors. Starting with a Crew Response Tree (CRT) that has identified human failure events, analysts identify potential failure mechanisms using the mid-layer model. The mid-layer model presented in this paper traces the identification of the failure mechanisms using the Information-Diagnosis/Decision-Action (IDA) model and cognitive models from the psychological literature. Each failure mechanism is grouped according to a phase of IDA. Under each phase of IDA, the cognitive models help identify the relevant performance shaping factors for the failure mechanism. The use of IDA and cognitive models can be traced through fault trees, which provide a detailed complement to the CRT.

During the 1990's the Electric Power Research Institute (EPRI) developed methods for fire risk analysis to support its utility members in the preparation of responses to Generic Letter 88-20, Supplement 4, "Individual Plant Examination - External Events" (IPEEE). This effort produced a Fire Risk Assessment methodology for at-power that was used by the majority of US Nuclear Power Plants (NPPs) in support of the IPEEE program and several NPPs oversees. Although these methods were acceptable for accomplishing the objectives of the IPEEE, EPRI and the U.S. Nuclear Regulatory Commission (NRC) recognized that these methods require upgrades to support current requirements for Risk-Informed/Performance-Based (RI/PB) applications. In 2001 EPRI and the NRC Office of Nuclear Regulatory Research (RES) embarked on a cooperative project to improve the state-of-the-art in fire risk assessment to support this new risk-informed environment in fire protection. This project produced a consensus document, NUREG/CR-6850 (EPRI 1011989), entitled "Fire PRA Methodology for Nuclear Power Facilities" which addresses fire risk for at-power operations. This report developed: 1) the process for identification and inclusion of the post-fire Human Failure Events (HFEs), 2) the methodology for assigning quantitative screening values to these HFEs, and 3) the initial considerations of performance shaping factors (PSFs) and related fire effects that may need to be addressed in developing best-estimate Human Error Probabilities (HEPs). However, this document does not describe a methodology to develop these best-estimate HEPs given the PSFs and the fire-related effects. In 2007 EPRI and NRC's RES embarked on another cooperative project to develop explicit guidance for estimating HEPs for human error events under fire generated conditions, building upon existing human reliability analysis (HRA) methods. This paper will describe the progress to date on the development and testing of the fire HRA methodology, which includes addressing the range of fire procedures used in existing plants, the range of strategies for main control room abandonment, and the potential impact of fire-induced spurious electrical effects on crew performance. In addition to developing a detailed HRA approach, one goal of the project is to develop a fire HRA scoping quantification approach that allows derivation of more realistic HEPs than those in the screening approach from NUREG/CR-6850 (EPRI 1011989), while requiring less analytic resources than a detailed HRA. In this approach, detailed HRA will be used only for the more complex actions that cannot meet the criteria for the scoping approach.

Abstract not provided.

An experiment was conducted comparing the effectiveness of individual versus group electronic brainstorming in order to address difficult, real world challenges. While industrial reliance on electronic communications has become ubiquitous, empirical and theoretical understanding of the bounds of its effectiveness have been limited. Previous research using short-term, laboratory experiments have engaged small groups of students in answering questions irrelevant to an industrial setting. The present experiment extends current findings beyond the laboratory to larger groups of real-world employees addressing organization-relevant challenges over the course of four days. Employees and contractors at a national security laboratory participated, either in a group setting or individually, in an electronic brainstorm to pose solutions to a 'wickedly' difficult problem. The data demonstrate that (for this design) individuals perform at least as well as groups in producing quantity of electronic ideas, regardless of brainstorming duration. However, when judged with respect to quality along three dimensions (originality, feasibility, and effectiveness), the individuals significantly (p<0.05) out-performed the group working together. When idea quality is used as the benchmark of success, these data indicate that work-relevant challenges are better solved by aggregating electronic individual responses, rather than electronically convening a group. This research suggests that industrial reliance upon electronic problem solving groups should be tempered, and large nominal groups might be the more appropriate vehicle for solving wicked corporate issues.

Abstract not provided.

Abstract not provided.

An experiment was conducted comparing the effectiveness of individual versus group electronic brainstorming in addressing real-world 'wickedly difficult' challenges. Previous laboratory research has engaged small groups of students in answering questions irrelevant to an industrial setting. The current experiment extended this research to larger, real-world employee groups engaged in addressing organization-relevant challenges. Within the present experiment, the data demonstrated that individuals performed at least as well as groups in terms of number of ideas produced and significantly (p < .02) outperformed groups in terms of the quality of those ideas (as measured along the dimensions of originality, feasibility, and effectiveness).

There is a diversity of human reliability analysis (HRA) methods available for use in assessing human performance within probabilistic risk assessment (PRA). Due to the significant differences in the methods, including the scope, approach, and underlying models, there is a need for an empirical comparison investigating the validity and reliability of the methods. To accomplish this empirical comparison, a benchmarking study is currently underway that compares HRA methods with each other and against operator performance in simulator studies. In order to account for as many effects as possible in the construction of this benchmarking study, a literature review was conducted, reviewing past benchmarking studies in the areas of psychology and risk assessment. A number of lessons learned through these studies are presented in order to aid in the design of future HRA benchmarking endeavors.

Abstract not provided.

An experiment was conducted comparing the effectiveness of individual versus group electronic brainstorming in order to address difficult, real world challenges. While industrial reliance on electronic communications has become ubiquitous, empirical and theoretical understanding of the bounds of its effectiveness have been limited. Previous research using short-term, laboratory experiments have engaged small groups of students in answering questions irrelevant to an industrial setting. The current experiment extends current findings beyond the laboratory to larger groups of real-world employees addressing organization-relevant challenges over the course of four days. Findings are twofold. First, the data demonstrate that (for this design) individuals perform at least as well as groups in producing quantity of electronic ideas, regardless of brainstorming duration. However, when judged with respect to quality along three dimensions (originality, feasibility, and effectiveness), the individuals significantly (p<0.05) out performed the group working together. The theoretical and applied (e.g., cost effectiveness) implications of this finding are discussed. Second, the current experiment yielded several viable solutions to the wickedly difficult problem that was posed.

Abstract not provided.