ProteusGPU

Proteus is a GPU accelerated moving mesh hydrodynamics code.

It combines the algorithmic approach of "Meshless Voronoi on the GPU" [Ray et. al 2018] with a moving mesh hydro solver similar to "AREPO" [Springel 2010] ported to GPU.

Note

The current version runs 2D/3D moving mesh hydrodynamics on NVIDIA GH200s as well as discrete NVIDIA GPUs and CPUs. A rudimentary multi-node version is implemented but optimizations are ongoing.

This project is being done during my master's thesis, supervised by Dylan Nelson, at the Institute of Theoretical Astrophysics, Heidelberg University.

Getting started

1. After cloning the repo select your system in Makefile.systype (or add your own to the Makefile)
2. Configure compilation flags in Config.sh and build with make
3. Use a create.py script for IC generation and specify simulation parameters in param.txt
4. Run the simulation with

./ProteusGPU [./ics/param.txt] [restart_flag]

If the restart_flag is set the simulation continues from the last snapshot in the output_folder.

Experimental: If you want to run the MPI version with additional multithreading try

OMP_NUM_THREADS=[threads_per_rank] mpirun -np [number_of_ranks] --bind-to none ./ProteusGPU

otherwise every mpi-rank uses only two threads per rank... (will figure out an improvement of that in the future)

Examples

Convergence of acoustic wave: second-order

Sod's shock tube test compared to AREPO

Taylor-Sedov blast wave (2D/3D):

Kelvin Helmholtz Instability ($51^2$ and $1024^2$)

Some colliding clouds ($200^3$, see ics/create_cloud_crash.py)

Runtime comparsion of Proteus (single GPU and single-node CPU) compared to AREPO (single-node CPU) for a $100^3$ shock tube test.

Note: here we estimate the runtime of AREPO runtime from its single core time, to eliminate its MPI overhead.

A first run with 4 MPI tasks (still in early development).
Seedpoints are colorcoded according to their rank.

Dependencies

• HDF5 (libhdf5-dev on Ubuntu, via Homebrew on macOS)
• CUDA Toolkit (for GPU mode, requires NVIDIA GPU)

Roadmap

• v0.1 - kNN and Voronoi mesh construction (2D and 3D, CPU)
• v0.2 - first order FV hydro (static mesh, 2D and 3D, CPU)
• v0.3 - second order FV hydro (static mesh, 2D and 3D, CPU)
• v0.4 - moving mesh hydro (2D, CPU)
• v0.5 - moving mesh hydro (3D, CPU)
• v0.6 - GPU initial port (moving mesh hydro)
• v0.7 - single GPU optimization
• v0.8 - multi-GPU (distributed memory, MPI)
• v0.9 - multi-GPU optimization
• v1.0 - support for inhomogenous particle distributions