Many teams struggle to adapt and right-size software engineering best practices for quality assurance to fit their context. Introducing software quality is not usually framed in a way that motivates teams to take action, thus resulting in it becoming a “check the box for compliance” activity instead of a cultural practice that values software quality and the effort to achieve it. When and how can we provide effective incentives for software teams to adopt and integrate meaningful and enduring software quality practices? Here, we explored this question through a persona-based ideation exercise at the 2021 Collegeville Workshop on Scientific Software in which we created three unique personas that represent different scientific software developer perspectives.
Although many software teams across the laboratories comply with yearly software quality engineering (SQE) assessments, the practice of introducing quality into each phase of the software lifecycle, or the team processes, may vary substantially. Even with the support of a quality engineer, many teams struggle to adapt and right-size software engineering best practices in quality to fit their context, and these activities aren’t framed in a way that motivates teams to take action. In short, software quality is often a “check the box for compliance” activity instead of a cultural practice that both values software quality and knows how to achieve it. In this report, we present the results of our 6600 VISTA Innovation Tournament project, "Incentivizing and Motivating High Confidence and Research Software Teams to Adopt the Practice of Quality." We present our findings and roadmap for future work based on 1) a rapid review of relevant literature, 2) lessons learned from an internal design thinking workshop, and 3) an external Collegeville 2021 workshop. These activities provided an opportunity for team ideation and community engagement/feedback. Based on our findings, we believe a coordinated effort (e.g. strategic communication campaign) aimed at diffusing the innovation of the practice of quality across Sandia National Laboratories could over time effect meaningful organizational change. As such, our roadmap addresses strategies for motivating and incentivizing individuals ranging from early career to seasoned software developers/scientists.
In response to the COVID-19 pandemic, the Exascale Computing Project’s (ECP) Interoperable Design of Extreme-scale Application Software (IDEAS) productivity team launched the panel series Strategies for Working Remotely to facilitate informal, cross-organizational dialog in the absence of face-to-face meetings. In a time of pandemic, organizations increasingly need to reach across perceived boundaries to learn from each other, so that we can move beyond stand-alone silos to more connected multidisciplinary and multiorganizational configurations. The present paper argues that the unplanned transition to remote work, overuse of electronic communication, and need to unlearn habits associated with an overreliance on face-to-face, created unique opportunities to learn from the situation and accelerate cross-institutional cooperation and collaboration through online community dialog facilitated by informal panel discussions. Recommendations for facilitating online panel discussions to foster cross-organizational dialog are provided by applying the Simulation Experience Design Method.
Productivity and Sustainability Improvement Planning (PSIP) is a lightweight, iterative workflow that allows software development teams to identify development bottlenecks and track progress to overcome them. In this paper, we present an overview of PSIP and how it compares to other software process improvement (SPI) methodologies, and provide two case studies that describe how the use of PSIP led to successful improvements in team effectiveness and efficiency.
One of the core missions of the Department of Energy (DOE) is to move beyond current high performance computing (HPC) capabilities toward a capable exascale computing ecosystem that accelerates scientific discovery and addresses critical challenges in energy and national security. The very nature of this mission has drawn a wide range of talented and successful scientists to work together in new ways to push beyond the status-quo toward this goal. For many scientists, their past success was achieved through efficient and agile collaboration within small trusted teams that rapidly innovate, prototype, and deliver. Thus, a key challenge for the ECP (Exascale Computing Project) is to scale this efficiency and innovation from small teams to aggregate teams of teams. While scaling agile collaboration from small teams to teams of teams may seem like a trivial transition, the path to exascale introduces significant uncertainty in HPC scientific software development for future modeling and simulation, and can cause unforeseen disruptions or inefficiencies that impede organizational productivity and innovation critical to achieving an integrated exascale vision. This paper identifies key challenges in scaling to a team of teams approach and recommends strategies for addressing them. The scientific community will take away lessons learned and recommended best practices from examples for enhancing productivity and innovation at scale for immediate use in modeling and simulation software engineering projects and programs.
Cross Reality (XR) immersive environments offer challenges and opportunities in designing for cognitive aspects (e.g. learning, memory, attention, etc.) of information design and interactions. Information design is a multidisciplinary endeavor involving data science, communication science, cognitive science, media, and technology. In the present paper the holodeck metaphor is extended to illustrate how information design practices and some of the qualities of this imaginary computationally augmented environment (a.k.a. the holodeck) may be achieved in XR environments to support information-rich storytelling and real life, face-to-face, and virtual collaborative interactions. The Simulation Experience Design Framework & Method is introduced to organize challenges and opportunities in the design of information for XR. The notion of carefully blending both real and virtual spaces to achieve total immersion is discussed as the reader moves through the elements of the cyclical framework. A solution space leveraging cognitive science, information design, and transmedia learning highlights key challenges facing contemporary XR designers. Challenges include but are not limited to interleaving information, technology, and media into the human storytelling process, and supporting narratives in a way that is memorable, robust, and extendable.
Marques, Osni A.; Bernholdt, David E.; Raybourn, Elaine M.; Barker, Ashley D.; Hartman-Baker, Rebecca J.
Here, in this contribution, we discuss our experiences organizing the Best Practices for HPC Software Developers (HPC-BP) webinar series, an effort for the dissemination of software development methodologies, tools and experiences to improve developer productivity and software sustainability. HPC-BP is an outreach component of the IDEAS Productivity Project [4] and has been designed to support the IDEAS mission to work with scientific software development teams to enhance their productivity and the sustainability of their codes. The series, which was launched in 2016, has just presented its 22nd webinar. We summarize and distill our experiences with these webinars, including what we consider to be "best practices" in the execution of both individual webinars and a long-running series like HPC-BP. We also discuss future opportunities and challenges in continuing the series.
Large-scale collaborative scientific software projects require more knowledge than any one person typically possesses. This makes coordination and communication of knowledge and expertise a key factor in creating and safeguarding software quality, without which we cannot have sustainable software. However, as researchers attempt to scale up the production of software, they are confronted by problems of awareness and understanding. This presents an opportunity to develop better practices and tools that directly address these challenges. To that end, we conducted a case study of developers of the Trilinos project. We surveyed the software development challenges addressed and show how those problems are connected with what they know and how they communicate. Based on these data, we provide a series of practicable recommendations, and outline a path forward for future research.
Sandia National Laboratories performed a 6-month effort to stand up a "zero-entry" cyber range environment for the purpose of providing self-directed practice to augment transmedia learning across diverse media and/or devices that may be part of a loosely coupled, distributed ecosystem. This 6-month effort leveraged Minimega, an open-source Emulytics™ (emulation + analytics) tool for launching and managing virtual machines in a cyber range. The proof of concept addressed a set of learning objectives for cybersecurity operations by providing three, short "zero-entry" exercises for beginner, intermediate, and advanced levels in network forensics, social engineering, penetration testing, and reverse engineering. Learners provided answers to problems they explored in networked virtual machines. The hands-on environment, Cyber Scorpion, participated in a preliminary demonstration in April 2017 at Ft. Bragg, NC. The present chapter describes the learning experience research and software development effort for a cybersecurity use case and subsequent lessons learned. It offers general recommendations for challenges which may be present in future learning ecosystems.
Mixed, augmented, and virtual reality holds promise for many securityrelated applications including physical security systems. When combined with models of a site, an augmented reality (AR) approach can be designed to enhance knowledge and understanding of the status of the facility. The present chapter describes how improved modeling and simulation will increase situational awareness by blurring the lines among the use of tools for analysis, rehearsal, and training-especially when coupled with immersive interaction experiences offered by augmented reality. We demonstrate how the notion of a digital twin can blur these lines. We conclude with challenges that must be overcome when applying digital twins, advanced modeling, and augmented reality to the design and development of next-generation physical security systems.
Raybourn, Elaine M.; Raybourn, Elaine M.; Schatz, Sae S.; Schatz, Sae S.; Volgol-Walcutt, Jennifer V.; Volgol-Walcutt, Jennifer V.; Vierling, Kendy V.; Vierling, Kendy V.
In this study, the authors define a new approach that addresses the challenge of efficiently designing informational artefacts for optimal knowledge acquisition, an important issue in cognitive ergonomics. Termed Industrial Knowledge Design (or InK'D), it draws from information-related (e.g. informatics) and neurosciences-related (e.g. neuroergonomics) disciplines. Although it can be used for a broad scope of communication-driven business functions, our focus as learning professionals is on conveying knowledge for purposes of training, education, and performance support. This paper discusses preliminary principles of InK'D practice that can be employed to maximise the quality and quantity of transferred knowledge through interaction design. The paper codifies tacit knowledge into explicit concepts that can be leveraged by expert and non-expert knowledge designers alike.
Next generation learning ecosystems will be comprised of intelligent, adaptive environments that utilize one’s cultural footprints to co-create shared narratives and facilitate intercultural understanding. The present paper discusses why digital footprints, cultural signposts, intercultural agents, and transmedia learning are needed to realize relevant learning in virtual environments. The paper introduces notions that may impact the design of culturally-aware information technology for distributed learning are presented.