Publications

56 Results
Skip to search filters

ADELUS: A Performance-Portable Dense LU Solver for Distributed-Memory Hardware-Accelerated Systems

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Dang, Vinh Q.; Kotulski, J.D.; Rajamanickam, Sivasankaran R.

Solving dense systems of linear equations is essential in applications encountered in physics, mathematics, and engineering. This paper describes our current efforts toward the development of the ADELUS package for current and next generation distributed, accelerator-based, high-performance computing platforms. The package solves dense linear systems using partial pivoting LU factorization on distributed-memory systems with CPUs/GPUs. The matrix is block-mapped onto distributed memory on CPUs/GPUs and is solved as if it was torus-wrapped for an optimal balance of computation and communication. A permutation operation is performed to restore the results so the torus-wrap distribution is transparent to the user. This package targets performance portability by leveraging the abstractions provided in the Kokkos and Kokkos Kernels libraries. Comparison of the performance gains versus the state-of-the-art SLATE and DPLASMA GESV functionalities on the Summit supercomputer are provided. Preliminary performance results from large-scale electromagnetic simulations using ADELUS are also presented. The solver achieves 7.7 Petaflops on 7600 GPUs of the Sierra supercomputer translating to 16.9% efficiency.

More Details

The parallel implementation and accuracy of matrix compression in the method of moments code EIGER

2018 International Applied Computational Electromagnetics Society Symposium in Denver, ACES-Denver 2018

Kotulski, J.D.

This paper describes the implementation of the adaptive cross approximation(ACA) in the method of moments code EIGER. This purely algebraic method provides a mechanism to reduce memory usage and overall computation time. In addition, this work has been targeted for massively parallel platforms to extend the viable frequency range for electromagnetic compatibility and interference problems.

More Details

Massively parallel frequency domain electromagnetic simulation codes

2018 International Applied Computational Electromagnetics Society Symposium in Denver, ACES-Denver 2018

Langston, William L.; Kotulski, J.D.; Coats, Rebecca S.; Jorgenson, Roy E.; Blake, S.A.; Campione, Salvatore; Pung, Aaron J.; Zinser, Brian

This paper provides an overview of the electromagnetic frequency domain simulation capabilities of the Electromagnetic Theory department at Sandia National Laboratories via a description of two of its codes. EIGER is a Method of Moments code for electromagnetic simulations, but it only runs on traditional CPUs, not on new architectures. Gemma is in development to replace EIGER and will run on many architectures, including CPUs, GPUs, and MICs, by leveraging the Kokkos library.

More Details

Performance portable sparse approximate inverse preconditioner for EFIE equations

Proceedings of the 2017 19th International Conference on Electromagnetics in Advanced Applications, ICEAA 2017

Bettencourt, Matthew T.; Zinser, Brian; Jorgenson, Roy E.; Kotulski, J.D.

A block base sparse approximate inverse preconditioner for the electric field integral equations is documented and tested. It utilized the Kokkos library for performance portability and shows superior performance when compared to a direct method, 36x faster for a 112K DOF problem. Furthermore, due to the abstractions available in the Kokkos library it allows one to migrate from CPU to GPU in a trivial way.

More Details

The effect of paramagnetic shift during thermal quench on internal components in fusion devices

Fusion Engineering and Design

Ulrickson, M.A.; Kotulski, J.D.

A plasma current disruption is usually initiated by impurity influx that causes a rapid decrease in plasma thermal stored energy (thermal quench). Thermal quench occurs in 500-2000 μs on a large device like ITER. Depending on the β value, the plasma may be either paramagnetic or diamagnetic. Thermal quench causes a large shift in paramagnetism (or diamagnetism) and a corresponding change in toroidal flux. The flux swing can be 1-2 Weber with the rate of change of the toroidal field between 25 and 150 T/s for a device like ITER. The toroidal field shift induces poloidal current in the vessel and possibly in internal components. We have developed a method for simulating the thermal quench field shift that is compatible for use with the electromagnetic simulation codes. The method is based on a radially thin shell having the shape of the last closed flux surface with poloidal current driven to duplicate the toroidal field shift. The magnitude of the current and its time history are adjusted to duplicate the flux change during a disruption thermal quench. We will present the results of using this method to simulate the induced currents in a vacuum vessel having two shells. © 2012 Elsevier B.V. All rights reserved.

More Details

Time harmonic two-dimensional cavity scar statistics: Convex mirrors and bowtie

Electromagnetics

Warne, Larry K.; Jorgenson, Roy E.; Kotulski, J.D.; Lee, K.S.H.

This article examines the localization of time harmonic high-frequency modal fields in two-dimensional cavities along periodic paths between opposing sides of the cavity. The cases where these orbits lead to unstable localized modes are known as scars. This article examines the enhancements for these unstable orbits when the opposing mirrors are convex, constructing the high-frequency field in the scar region using elliptic cylinder coordinates in combination with a random reflection phase from the outer chaotic region. The enhancements when the cavity is symmetric as well as asymmetric about the orbit are examined. © Taylor & Francis Group, LLC.

More Details

Intelligent front-end sample preparation tool using acoustic streaming

Vreeland, Erika C.; Smith, Gennifer T.; Edwards, Thayne L.; James, Conrad D.; McClain, Jaime L.; Murton, Jaclyn K.; Kotulski, J.D.; Clem, Paul G.

We have successfully developed a nucleic acid extraction system based on a microacoustic lysis array coupled to an integrated nucleic acid extraction system all on a single cartridge. The microacoustic lysing array is based on 36{sup o} Y cut lithium niobate, which couples bulk acoustic waves (BAW) into the microchannels. The microchannels were fabricated using Mylar laminates and fused silica to form acoustic-fluidic interface cartridges. The transducer array consists of four active elements directed for cell lysis and one optional BAW element for mixing on the cartridge. The lysis system was modeled using one dimensional (1D) transmission line and two dimensional (2D) FEM models. For input powers required to lyse cells, the flow rate dictated the temperature change across the lysing region. From the computational models, a flow rate of 10 {micro}L/min produced a temperature rise of 23.2 C and only 6.7 C when flowing at 60 {micro}L/min. The measured temperature changes were 5 C less than the model. The computational models also permitted optimization of the acoustic coupling to the microchannel region and revealed the potential impact of thermal effects if not controlled. Using E. coli, we achieved a lysing efficacy of 49.9 {+-} 29.92 % based on a cell viability assay with a 757.2 % increase in ATP release within 20 seconds of acoustic exposure. A bench-top lysing system required 15-20 minutes operating up to 58 Watts to achieve the same level of cell lysis. We demonstrate that active mixing on the cartridge was critical to maximize binding and release of nucleic acid to the magnetic beads. Using a sol-gel silica bead matrix filled microchannel the extraction efficacy was 40%. The cartridge based magnetic bead system had an extraction efficiency of 19.2%. For an electric field based method that used Nafion films, a nucleic acid extraction efficiency of 66.3 % was achieved at 6 volts DC. For the flow rates we tested (10-50 {micro}L/min), the nucleic acid extraction time was 5-10 minutes for a volume of 50 {micro}L. Moreover, a unique feature of this technology is the ability to replace the cartridges for subsequent nucleic acid extractions.

More Details

Electromagnetic analysis of transient disruption forces on the ITER shield modules

Fusion Engineering and Design

Kotulski, J.D.; Coats, Rebecca S.; Pasik, Michael F.

This paper describes the eddy current computation and the resultant forces and torques on selected shield modules assigned to the US team that occur due to plasma disruption. The plasma disruption considered is referred to as major disruption (MD) and is one of the disruption cases defined by the International Organization (IO). This paper identifies the applicability of geometrical simplifications for future design analyses. In particular it is shown that cutting a module in half does not preserve the physics of the eddy current generation and resultant calculations while modeling a full module including the nearest modules does preserve the fundamental physics. The force results are shown for shield modules 7 and 13 exposing the validity of geometrical simplifications. The computed torque for these two modules is also presented. © 2008 Elsevier B.V.

More Details

Two dimensional unstable scar statistics

Warne, Larry K.; Jorgenson, Roy E.; Kotulski, J.D.

This report examines the localization of time harmonic high frequency modal fields in two dimensional cavities along periodic paths between opposing sides of the cavity. The cases where these orbits lead to unstable localized modes are known as scars. This paper examines the enhancements for these unstable orbits when the opposing mirrors are both convex and concave. In the latter case the construction includes the treatment of interior foci.

More Details

Adaptive mesh refinement for time-domain electromagnetics using vector finite elements :a feasibility study

Pasik, Michael F.; Kotulski, J.D.; Turner, C.D.

This report investigates the feasibility of applying Adaptive Mesh Refinement (AMR) techniques to a vector finite element formulation for the wave equation in three dimensions. Possible error estimators are considered first. Next, approaches for refining tetrahedral elements are reviewed. AMR capabilities within the Nevada framework are then evaluated. We summarize our conclusions on the feasibility of AMR for time-domain vector finite elements and identify a path forward.

More Details

A set of verification test cases for Eiger : plane wave scattering from a sphere

Jorgenson, Roy E.; Kotulski, J.D.

This report discusses a set of verification test cases for the frequency-domain, boundary-element, electromagnetics code Eiger based on the analytical solution of plane wave scattering from a sphere. Three cases will be considered: when the sphere is made of perfect electric conductor, when the sphere is made of lossless dielectric and when the sphere is made of lossy dielectric. We outline the procedures that must be followed in order to carefully compare the numerical solution to the analytical solution. We define an error criterion and demonstrate convergence behavior for both the analytical and numerical cases. These problems test the code's ability to calculate the surface current density and secondary quantities, such as near fields and far fields.

More Details

Electromagnetic analysis and modeling of the coax-to-triplate transition for the pulse-compression section of the ZR accelerator

Digest of Technical Papers-IEEE International Pulsed Power Conference

Johnson, William Arthur.; Coats, Rebecca S.; Jorgenson, Roy E.; Kotulski, J.D.; Lehr, J.M.; Pasik, Michael F.; Rosenthal, Stephen E.; Turner, C.D.; Warne, Larry K.

Transverse electromagnetic (TEM) wave analysis is used to estimate the efficiencies of the coax to triplate transition in Sandia's Z-20 test module. The structure of both the TEM mode and higher order TE modes in the triplate transmission line are characterized. In addition, three dimensional time domain simulations are carried out and used in conjunction with the modal analysis to provide insight into the wave structure excited in the triplate transmission line.

More Details
56 Results
56 Results