Uploaded November 2022 | Updated September 2026, 1 week ago

---

The wheels are now rotating in this simulation, via moving bounce-back boundaries. Settling a debate, started by my other video of the F1 car where wheels were not rotating: youtu.be/q4rNIbqvyQI
How much does wheel rotation affect airflow? When the wheels rotate, the turbulent boundary layer detaches a bit earlier from the tire surface. This creates significantly more turbulence.

FluidX3D source code: github.com/ProjectPhysX/FluidX3D

The 10 seconds of video show the car driving at 100km/h for 0.5 seconds. Moving boundaries impose no additional computational cost. This took 7 hours to compute+render on 8x AMD Instinct MI200 GPUs. Grid resolution is 2144×4288×1072, and 108360 time steps are computed, with 3x 4K images rendered every 180 LBM time steps. Shown is velocity-colored Q-criterion isosurfaces with marching-cubes. Reynolds number is 3.75 Million with Smagorinsky-Lilly subgrid model.

How is it possible to squeeze 10 billion grid points in only 512GB VRAM?
I'm using two techniques here, which together form the holy grail of lattice Boltzmann, cutting memory demand down to only 55 Bytes/node for D3Q19 LBM, or 1/3 of conventional codes:

1. In-place streaming with Esoteric-Pull. This almost cuts memory demand in half and slightly increases performance due to implicit bounce-back boundaries.
Paper: doi.org/10.3390/computation10060092

2. Decoupled arithmetic precision (FP32) and memory precision (FP16): all arithmetic is done in FP32, but LBM density distribution functions in memory are compressed to FP16. This almost cuts memory demand in half and almost doubles performance, without impacting overall accuracy for most setups.
Paper: researchgate.net/publication/362275548_Accuracy_and_performance_of_the_lattice_Boltzmann_method_with_64-bit_32-bit_and_customized_16-bit_number_formats

Graphics are done directly in FluidX3D with OpenCL, with the raw simulation data already residing in ultra-fast video memory. 1 frame of the velocity field is 118GB, 603 frames are generated, which would be 71TB. No volumetric data ever has to be copied to the CPU or hard drive, but only rendered 4K frames (33MB) instead. Once on the CPU side, a copy of the frame is made in memory and a thread is detached to handle the slow .png compression, all while the simulation is already continuing. At any time, about 16 frames are compressed in parallel on 16 CPU cores, while the simulation is running on GPU.
Paper: researchgate.net/publication/360501260_Combined_scientific_CFD_simulation_and_interactive_raytracing_with_OpenCL

Timestamps:
0:00 side view
0:10 front view
0:20 top view

Thanks to the people at Jülich Supercomputing Centre for letting me test their hardware!
The 3D model is from Thingiverse: thingiverse.com/thing:2990512/files

#CFD #GPU #FluidX3D #OpenCL