The magnetohydrodynamics (MHD) model describes the dynamics of charged fluids in the presence of electromagnetic fields. MHD models are used to describe important phenomena in the natural world (e.g., solar flares, astrophysical magnetic field generation, Earth’s magnetosphere interaction with the solar wind) and in technological applications (e.g., spacecraft propulsion, magnetically confined plasma for fusion energy devices such as tokamak reactors (e.g. ITER), and plasma dynamics in pulsed reactors such as Sandia’s Z-pinch device). We have an active research program to develop advanced computational formulations and solution methods for multiphysics MHD, and we deploy the results of this research in our large-scale massively parallel MHD simulation codes.
The mathematical basis for the continuum modeling of MHD systems is the solution of the governing partial differential equations (PDEs) describing conservation of mass, momentum, and energy, augmented by Maxwell’s equations for the electric and magnetic field. This system of PDEs is non-self adjoint, strongly coupled, highly nonlinear, and characterized by multiple physical phenomena that span a very large range of length- and time-scales. These interacting, nonlinear multiple time-scale physical mechanisms can balance to produce steady-state behavior, nearly balance to evolve a solution on a dynamical time scale that is long relative to the component time-scales, or can be dominated by just a few fast modes. These characteristics make the scalable, robust, accurate, and efficient computational solution of these systems over relevant dynamical time scales of interest extremely challenging.
Our production MHD capabilities are contained within a family of multiphysics codes known as ALEGRA*. The codes — including ALEGRA, ALEGRA-MHD, ALEGRA-HEDP and ALEGRA-EMMA — constitute an extensive set of physics modeling capabilities built on software in the Nevada framework and third-party libraries. They simulate large deformations and strong shock physics including solid dynamics in an Arbitrary Lagrangian-Eulerian methodology as well as magnetics, magnetohydrodynamics, electromechanics and a wide range of phenomena for high-energy physics applications. Our principal customers have applied these codes in a variety of Z-pinch physics experiment designs and applications, in the development of advanced armor concepts, and in numerous National Security programs. Research and development in advanced methods, including code frameworks, large scale inline meshing, multiscale lagrangian hydrodynamics, resistive magnetohydrodynamic methods, material interface reconstruction, and code verification and validation, keeps the software on the cutting edge of high performance computing.
We also conduct research and development of advanced computational formulations and solution methods for challenging multiple-time-scale multiphysics MHD systems. For multiple-time-scale systems, fully-implicit methods can be an attractive choice that can often provide unconditionally-stable time integration techniques. The stability of these methods, however, comes at a cost, as these techniques generate large and highly nonlinear sparse systems of equations that must be solved at each time step. In the context of MHD, the dominant computational solution strategy has been the use of explicit, semi-implicit, and operator-splitting time integration methods. With the exception of fully-explicit strategies, which are limited by severe stability restrictions to follow the fastest component time scale, all these temporal integration methods include some implicitness to enable a more efficient solution of MHD systems. Such implicitness is aimed at removing one or more sources of numerical stiffness in the problem, either from parabolic diffusion or from fast wave phenomena. While these types of techniques currently form the basis for most production-level resistive MHD simulation tools, a number of outstanding numerical and computational issues remain. These include conditional stability limits, operator-splitting-type errors, heuristic time-step-controls and limited temporal orders of accuracy.
In our DOE Advanced Scientific Computing Research Applied Math Program funded research** we are pursuing the development and evaluation of
- higher-order-accurate, scalable, and efficient fully-implicit formulations for resistive and extended MHD with coupled multiphysics effects (e.g. anisotropic transport, multiple temperatures, coupled radiation-diffusion models, etc.),
- stable and accurate spatial discretizations based on unstructured mesh FE approximations that allow efficient enforcement of physical constraints (e.g. conservation, positivity preservation, div B = 0, etc),
- strongly coupled Newton-Krylov nonlinear solver with new physics-based and approximate block factorization preconditioners that enable scalable multilevel sub-block solvers,
- new mathematical algorithms and computer science techniques to effectively utilize extreme-scale resources with very high-core counts and high-concurrency node architectures.
To enable the research described above SNL has developed a very flexible multiphysics MHD simulation code, Drekar::XMHD that enables the development of new multiphysics MHD models and allows the rapid prototyping of new computational formulations and solution methods on large-scale parallel machines. This code has been demonstrated to weak-scale on a Cray XK7 and an IBM BG/Q on up to 128K and 256K cores (respectively). Recently we have also carried out strong scaling studies on up to 500,000 cores of an IBM BG/Q for a fully-coupled Krylov/AMG V-cycle linear solver that is a critical kernel for scalable solution of MHD systems.
