Publications

Model validation efforts often use a suite of experiments to provide data to test models for predictive use for a targeted application. A question that naturally arises is Does the experimental suite provide data to adequately test the target application model? The goal of this article is to develop methodology to partially address this question. The methodology utilizes computational models for the individual test suite experiments and for the target application, to assess coverage. The impact of uncertainties in model parameters on the assessment is addressed. Simple linear and nonlinear heat conduction examples of the methodology are provided. Copyright © Taylor & Francis Group, LLC.

The objective of this Standard is the specification of a verification and validation approach that quantifies the degree of accuracy inferred from the comparison of solution and data for a specified variable at a specified validation point. The approach uses the concepts from experimental uncertainty analysis to consider the errors and uncertainties in both the solution and the data. The scope of this Standard is the quantification of the degree of accuracy of simulation of specified validation variables at a specified validation point for cases in which the conditions of the actual experiment are simulated. Consideration of solution accuracy at points within a domain other than the validation points, for example interpolation/extrapolation in a domain of validation, is a matter of engineering judgment specific to each family of problems and is beyond the scope of this Standard.

The political dynamics associated with an election are typically a function of the interplay between political leaders and voters, as well as endogenous and exogenous factors that impact the perceptions and goals of the electorate. This paper describes an effort by Sandia National Laboratories to model the attitudes and behaviors of various political groups along with that population's primary influencers, such as government leaders. To accomplish this, Sandia National Laboratories is creating a hybrid system dynamics-cognitive model to simulate systems- and individual-level political dynamics in a hypothetical society. The model is based on well-established psychological theory, applied to both individuals and groups within the modeled society. Confidence management processes are being incorporated into the model design process to increase the utility of the tool and assess its performance. This project will enhance understanding of how political dynamics are determined in democratic society.

Abstract not provided.

Planning is an important part of computational model verification and validation (V&V) and the requisite planning document is vital for effectively executing the plan. The document provides a means of communicating intent to the typically large group of people, from program management to analysts to test engineers, who must work together to complete the validation activities. This report provides guidelines for writing a validation plan. It describes the components of such a plan and includes important references and resources. While the initial target audience is the DART Full System Model teams in the nuclear weapons program, the guidelines are generally applicable to other modeling efforts. Our goal in writing this document is to provide a framework for consistency in validation plans across weapon systems, different types of models, and different scenarios. Specific details contained in any given validation plan will vary according to application requirements and available resources.

Computational and mathematical models are developed in engineering to represent the behavior of physical systems to various system inputs and conditions. These models are often used to predict at other conditions, rather than to just reproduce the behavior of data obtained at the experimental conditions. For example, the boundary or initial conditions, time of prediction, geometry, material properties, and other model parameters can be different at test conditions than those for an anticipated application of a model. Situations for which the conditions may differ include those for which (1) one is in the design phase and a prototype of the system has not been constructed and tested under the anticipated conditions, (2) only one version of a final system can be built and destructive testing is not feasible, or (3) the anticipated design conditions are variable and one cannot easily reproduce the range of conditions with a limited number of carefully controlled experiments. Because data from these supporting experiments have value in model validation, even if the model was tested at different conditions than an anticipated application, methodology is required to evaluate the ability of the validation experiments to resolve the critical behavior for the anticipated application. The methodology presented uses models for the validation experiments and a model for the application to address how well the validation experiments resolve the application. More specifically, the methodology investigates the tradeoff that exists between the uncertainty (variability) in the behavior of the resolved critical variables for the anticipated application and the ability of the validation experiments to resolve this behavior. The important features of this approach are demonstrated through simple linear and non-linear heat conduction examples.

Two major issues associated with model validation are addressed here. First, we present a maximum likelihood approach to define and evaluate a model validation metric. The advantage of this approach is it is more easily applied to nonlinear problems than the methods presented earlier by Hills and Trucano (1999, 2001); the method is based on optimization for which software packages are readily available; and the method can more easily be extended to handle measurement uncertainty and prediction uncertainty with different probability structures. Several examples are presented utilizing this metric. We show conditions under which this approach reduces to the approach developed previously by Hills and Trucano (2001). Secondly, we expand our earlier discussions (Hills and Trucano, 1999, 2001) on the impact of multivariate correlation and the effect of this on model validation metrics. We show that ignoring correlation in multivariate data can lead to misleading results, such as rejecting a good model when sufficient evidence to do so is not available.

This report describes the underlying principles and goals of the Sandia ASCI Verification and Validation Program Validation Metrics Project. It also gives a technical description of two case studies, one in structural dynamics and the other in thermomechanics, that serve to focus the technical work of the project in Fiscal Year 2001.