Software architecture. We will design the software framework to combine various components, including Exasim, ΣMIT, and new software components, in a flexible and modular fashion. The coupling between Exasim and ΣMIT will be the core simulation. To develop MD potentials and MD data (particularly for the QK gas-surface chemistry model and the constitutive model for continuum simulation), we will utilize LAMMPS for MD, E3NN for ENNs, and Quantum-ESPRESSO for DFT. We will also use DSMC via SPARTA to validate Exasim’s hypersonic flow.
Outside of these other codes though, our main code will first utilize software such as PETSc, Trillinos, preCICE, and bespoke Julia packages to build a partitioned multi-physics simulation from Exasim and ΣMIT while utilizing other CS tools within one of the packages, the coupling, or UQ. This design strategy aims to take advantage of the capabilities and performance of Exasim and ΣMIT while using new software in the context of these existing codes and their coupling. We will aim to use our DSLs and exascale technologies to augment and improve the capabilities of the simulation software, by improving the performance and portability of the simulation on exascale hardware and by allowing new software for physical models to be developed.
As an important initial goal, we will develop GPU capabilities for key parts of ΣMIT. We will initially pursue this by combining ΣMIT’s existing modular design and PETSc support with a portable kernel language, such as Kokkos or OpenCilk/Kitsune. Later, we plan to leverage a GPU-capable FEM and ENN DSL that adapts to ΣMIT’s existing design choices, specifically, its data distribution.
Similarly, as UQ capabilities are built up, we will use Enzyme within the simulation for UQ. We plan to first use Enzyme directly, and then later integrate Enzyme with DSLs to take accurate mathematically non-trivial derivatives. Later in the project, we will leverage common compiler infrastructure (LLVM, MLIR, OpenCilk/Kitsune, Polygeist) to aid the coupled simulation, the DSLs, and the application of Enzyme. We will aim to make the incorporation of DSL codes, Julia libraries, and compiler technologies to be modular, to adapt to the existing software, and to be opt-in rather than forced on users. This aim necessitates careful design, documentation, and collaboration at software interfaces.
Development, maintenance, and open source. While different groups are responsible for developing, supporting, and maintaining different software components within different repositories, the overall simulation will be managed in a central Git repository. In particular, the documentation and tests within this repository will be a focus of col- laboration between subgroups developing separate software components. The documentation, modularity, tests (unit, performance, full system, and regression), and continuous integration (CI) will be a locus of collaboration in addition to good software practice. Different groups will develop their own projects as standalone systems with standalone releases, documentation, tests and CI.
Meanwhile, the central repository will unify these systems and provide its own releases, documentation, tests, and CI to support collaboration between groups. The software architect, PI Nguyen, will maintain the central repository and ensure this mode of collaboration is followed. We intend to make as much of the simulation software stack publicly available and open source as possible. In particular, we plan to make all of the exascale computing technologies we will develop freely available and open source.