1 Billion Cell Raindrop Impact CFD Simulation @ProjectPhysX
1 Billion Cell Raindrop Impact CFD Simulation  @ProjectPhysX
Uploaded September 2022 | Updated November 2026, 3 hours ago
FluidX3D source code: github.com/ProjectPhysX/FluidX3D

Full video in 4K: youtu.be/VadLwt9OqMo
Technical talk on how I did it: youtu.be/pD8JWAZ2f8o

This is a 1 billion grid cell simulation of a d = 7 mm diameter terminal velocity (9.55 m/s) raindrop impact at 20° inclination, simulated for 3 milliseconds (17355 LBM time steps). The box is 5d x 5d x 4.25d and the pool height is 2d. For reproducibility: The dimensionless numbers for this setup are Re = 63618, We = 8864, Fr = 36.44, Ca = 0.1393, Bo = 6.674.
Velocity, density and fluid mass are solved at 1 billion lattice points (1060 x 1060 x 900) with D3Q19 SRT and FP16S memory compression. The simulation requires 64 GB video memory and was conducted on an AMD Instinct MI200 GPU at Jülich Supercomputing Centre (JSC) JURECA-DC-MI200. Compute time alone was 61 minutes, plus 64 minutes rendering time (raytracing at 4K60 resolution with 1-10 rays per pixel; 4x 446 frames (11.8GB) generated).

Find physical accuracy validation of the raindrop setup in this paper: doi.org/10.1186/s43591-021-00018-8

How is it possible to squeeze 1 billion grid points in only 64GB VRAM?
I'm using two techniques here, which together form the holy grail of lattice Boltzmann, cutting memory demand down to only 67 Bytes/node for D3Q19 LBM with the Volume-of-Fluid extension, or 1/3 of conventional codes:

1. In-place streaming with Esoteric-Pull. This almost cuts memory demand in half and slightly increases performance due to implicit bounce-back boundaries.
Paper: doi.org/10.3390/computation10060092

2. Decoupled arithmetic precision (FP32) and memory precision (FP16): all arithmetic is done in FP32, but LBM density distribution functions in memory are compressed to FP16. This almost cuts memory demand in half and almost doubles performance, without impacting overall accuracy for most setups.
Paper: researchgate.net/publication/362275548_Accuracy_and_performance_of_the_lattice_Boltzmann_method_with_64-bit_32-bit_and_customized_16-bit_number_formats

Graphics are done directly in FluidX3D with OpenCL, with the raw simulation data already residing in ultra-fast video memory. No volumetric data ever has to be copied to the CPU or hard drive, but only rendered frames instead. Once on the CPU side, a copy of the frame is made in memory and a thread is detached to handle the slow .png compression, all while the simulation is already continuing. At any time, about 16 frames are compressed in parallel on 16 CPU cores, while the simulation is running on GPU.
Paper: researchgate.net/publication/360501260_Combined_scientific_CFD_simulation_and_interactive_raytracing_with_OpenCL

#shorts
1 Billion Cell Raindrop Impact CFD SimulationPhysX3D [GRAVITY SIMULATION] Der zweite Mond der Erde – 2016 HO31 Billion Cell Raindrop Impact CFD Simulation, 2 hours on 1 AMD Instinct MI200 64GB GPUFluidX3D [FLUID SIMULATION] Real Time Prototype DemoAerodynamics of a cowInsect inside LCD Display?!PhysX3D [GRAVITY SIMULATION] Galaxy fly-through + Blooming Effekt demoIndefinite magnetic levitation at room temperature (diamagnetic Bismuth)SpaceX Starship Belly Flop CFD Simulation in 160GB GPU VRAMReal-Time Fluid Raytracing with OpenCL (on an old GTX GPU)CFD timelapse of wind over citySimulating terminal velocity raindrop impacts with FluidX3D at Reynolds number 51k
Dr. Moritz Lehmann |

1 Billion Cell Raindrop Impact CFD Simulation

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER