The Grainger College of Engineering
Parallel Programming Laboratory

Center for Simulation of Advanced Rockets (CSAR): Software Framework for Advanced Rocket Simulation

The Center for Simulation of Advanced Rockets (CSAR) is one of five university-based research centers funded by the U.S. Department of Energy as part of its Advanced Simulation and Computing (ASCI) program. The principal goal of CSAR is detailed, integrated, whole-system simulation of solid propellant rockets under both normal and abnormal operating conditions.

Our research at PPL is focused on application of object-based parallel programming paradigms for computational science and engineering. Past research in our group allows building multi-paradigm adaptive parallel programs based on the Converse runtime system and Charm++ load-balancing framework. We have adapted this framework to procedural languages such as Fortran 90, and message passing programming paradigms. Detailed performance analysis of Adaptive MPI, an implementation of MPI standard on top of Charm++, has been done, and has been optimized to have minimal overhead as compared to native MPI on several machines. Various components of the integrated rocket simulation codes, ROCFLO, ROCSOLID, and ROCFACE have been ported to Adaptive MPI with minimal changes and with little effort, providing them with automatic load balancing capabilities. The AMPI implementation also enhances the modularity of rocket codes by using separate sets of virtual processors for ROCFLO and ROCSOLID. Based on a similar programming paradigm, a prototype finite element method (FEM) framework has been developed. Although originally designed for finite element applications, it is general enough for unstructured mesh applications. Using the FEM framework, code developers write only sequential code that implements the node and element-specific computations of the FEM solver, while the details of parallelism such as partitioning, communication, and load balancing are taken care of by the framework. A 2-D pressure-driven crack propagation application (originally developed by P. Geubelle et al) has been ported to use this framework.

The multi-partitioned object-based approach has several other benefits besides automatic load balancing and efficient cache reuse. We have developed prototype software components that allow parallel programs to shrink and expand at runtime in order to effectively use inherently dynamic computational resources, such as clusters of workstations (COWs), which are becoming increasingly popular parallel architectures. Also, an efficient checkpointing framework has been developed that allows a parallel program to checkpoint and restart on possibly different number of processors.

We are also developing a Parallel Mesh Adaptation Framework for use with our FEM framework. These codes perform mesh refinement/derefinement and quality improvements in parallel.

People

Papers / Talks