Performance portability is a challenge for application developers as the source code needs to be executed and performant on various hybrid computing architectures. The linear iterative solvers implemented in most applications consume more than 70% of the runtime. This paper presents the results of a linear solver in Trilinos for fluidized bed applications. The linear solver implemented in our code is based on the Kokkos programming model in Trilinos, which uses a library approach to provide performance portability across diverse devices with different memory models. For large scale problems, the numerical experiments on Xeon Phi and Kepler GPU architectures show good performance over the results on Xeon (Haswell) computing architectures.
MFiX, a general-purpose Fortran-based suite, simulates the complex flow in fluidized bed applications via BiCGStab and GMRES methods along with plane relaxation preconditioners. Trilinos, an object-oriented framework, contains various first- and second-generation Krylov subspace solvers and preconditioners. We developed a framework to integrate MFiX with Trilinos as MFiX does not possess advanced linear methods. The framework allows MFiX to access advanced linear solvers and preconditioners in Trilinos. The integrated solver is called MFiX–Trilinos, here after. In the present work, we study the performance of variants of GMRES and CGS methods in MFiX–Trilinos and BiCGStab and GMRES solvers in MFiX for a 3D gas–solid fluidized bed problem. Two right preconditioners employed along with various solvers in MFiX–Trilinos are Jacobi and smoothed aggregation. The flow from MFiX–Trilinos is validated against the same from MFiX for BiCGStab and GMRES methods. And, the effect of the preconditioning on the iterative solvers in MFiX–Trilinos is also analyzed. In addition, the effect of left and right smoothed aggregation preconditioning on the solvers is studied. The performance of the first- and second-generation solver stacks in MFiX–Trilinos is studied as well for two different problem sizes.
American Society of Mechanical Engineers, Fluids Engineering Division (Publication) FEDSM
Krushnarao Kotteda, V.M.; Kumar, Vinod; Chattopadhyay, Ashesh; Spotz, William S.
A framework is developed to integrate MFiX (Multiphase Flow with Interphase eXchanges) with advanced linear solvers in Trilinos. MFiX is a widely used open source general purpose multiphase solver developed by National Energy Technology Laboratories and written in Fortran. Trilinos is an objectedoriented open source software development platform from Sandia National Laboratories for solving large scale multiphysics problems. The framework handles the different data structures in Fortran and C++ and exchanges the information from MFiX to Trilinos and vice versa. The integrated solver, called MFiXTrilinos hereafter, provides next-generation computational capabilities including scalable linear solvers for distributed memory massively parallel computers. In this paper, the solution from the standard linear solvers in MFiX-Trilinos is validated against the same from MFiX for 2D and 3D fluidized bed problems. The standard iterative solvers considered in this work are BiConjugate Gradient Stabilized (BiCGStab) and Generalized minimal residual methods (GMRES) as the matrix is non-symmetric in nature. The stopping criterion set for the iterative solvers is same. It is observed that the solution from the integrated solver and MFiX is in good agreement.
ASME International Mechanical Engineering Congress and Exposition, Proceedings (IMECE)
Chattopadhyay, Ashesh; Kotteda, V.M.Krushnarao; Kumar, Vinod; Spotz, William S.
A framework is developed to integrate the existing MFiX (Multiphase Flow with Interphase eXchanges) flow solver with state-of-the-art linear equation solver packages in Trilinos. The integrated solver is tested on various flow problems. The performance of the solver is evaluated on fluidized bed problems and observed that the integrated flow solver performs better compared to the native solver.
PyTrilinos is a set of Python interfaces to compiled Trilinos packages. This collection supports serial and parallel dense linear algebra, serial and parallel sparse linear algebra, direct and iterative linear solution techniques, algebraic and multilevel preconditioners, nonlinear solvers and continuation algorithms, eigensolvers and partitioning algorithms. Also included are a variety of related utility functions and classes, including distributed I/O, coloring algorithms and matrix generation. PyTrilinos vector objects are compatible with the popular NumPy Python package. As a Python front end to compiled libraries, PyTrilinos takes advantage of the flexibility and ease of use of Python, and the efficiency of the underlying C++, C and Fortran numerical kernels. This paper covers recent, previously unpublished advances in the PyTrilinos package.
The fundamental ideas of the high order compact method are combined with the generalized finite difference method. The result is a finite difference method that works on unstructured, nonuniform grids, and is more accurate than one would classically expect from the number of grid points employed.
This report describes the activities and results of a Sandia LDRD project whose objective was to develop and demonstrate foundational aspects of a next-generation nuclear reactor safety code that leverages advanced computational technology. The project scope was directed towards the systems-level modeling and simulation of an advanced, sodium cooled fast reactor, but the approach developed has a more general applicability. The major accomplishments of the LDRD are centered around the following two activities. (1) The development and testing of LIME, a Lightweight Integrating Multi-physics Environment for coupling codes that is designed to enable both 'legacy' and 'new' physics codes to be combined and strongly coupled using advanced nonlinear solution methods. (2) The development and initial demonstration of BRISC, a prototype next-generation nuclear reactor integrated safety code. BRISC leverages LIME to tightly couple the physics models in several different codes (written in a variety of languages) into one integrated package for simulating accident scenarios in a liquid sodium cooled 'burner' nuclear reactor. Other activities and accomplishments of the LDRD include (a) further development, application and demonstration of the 'non-linear elimination' strategy to enable physics codes that do not provide residuals to be incorporated into LIME, (b) significant extensions of the RIO CFD code capabilities, (c) complex 3D solid modeling and meshing of major fast reactor components and regions, and (d) an approach for multi-physics coupling across non-conformal mesh interfaces.
PyTrilinos provides python access to selected Trilinos packages: emerging from early stages, portability, completeness; parallelism; rapid prototyping; application development; unit testing; and numeric compatibility (migrating to NumPy). PyTrilinos complements and supplements the SciPy package.