Sandia National Laboratories (SNL) is designing and developing an Artificial Intelligence (AI)-enabled smart digital assistant (SDA), Inspecta (International Nuclear Safeguards Personal Examination and Containment Tracking Assistant). The goal is to provide inspectors an in-field digital assistant that can perform tasks identified as tedious, challenging, or prone to human error. During 2021, we defined the requirements for Inspecta based on reviews of International Atomic Energy Agency (IAEA) publications and interviews with former IAEA inspectors. We then mapped the requirements to current commercial or open-source technical capabilities to provide a development path for an initial Inspecta prototype while highlighting potential research and development tasks. We selected a highimpact inspection task that could be performed by an early Inspecta prototype and are developing the initial architecture, including hardware platform. This paper describes the methodology for selecting an initial task scenario, the first set of Inspecta skills needed to assist with that task scenario and finally the design and development of Inspecta’s architecture and platform.
Reverse engineering (RE) analysts struggle to address critical questions about the safety of binary code accurately and promptly, and their supporting program analysis tools are simply wrong sometimes. The analysis tools have to approximate in order to provide any information at all, but this means that they introduce uncertainty into their results. And those uncertainties chain from analysis to analysis. We hypothesize that exposing sources, impacts, and control of uncertainty to human binary analysts will allow the analysts to approach their hardest problems with high-powered analytic techniques that they know when to trust. Combining expertise in binary analysis algorithms, human cognition, uncertainty quantification, verification and validation, and visualization, we pursue research that should benefit binary software analysis efforts across the board. We find a strong analogy between RE and exploratory data analysis (EDA); we begin to characterize sources and types of uncertainty found in practice in RE (both in the process and in supporting analyses); we explore a domain-specific focus on uncertainty in pointer analysis, showing that more precise models do help analysts answer small information flow questions faster and more accurately; and we test a general population with domain-general sudoku problems, showing that adding "knobs" to an analysis does not significantly slow down performance. This document describes our explorations in uncertainty in binary analysis.