Chevrolet C10 1961 in 4.3 Billion cell CFD - FluidX3D on 2x Intel Xeon 6979P @ProjectPhysX

Chevrolet C10 1961 in 4.3 Billion cell CFD - FluidX3D on 2x Intel Xeon 6979P @ProjectPhysX

Uploaded November 2024 | Updated September 2026, 2 weeks ago

#SC24 day 2! Here's a "smaller" #FluidX3D #CFD simulation, the Chevrolet C10 1961, at 4.3 Billion cells. Probably one of the most un-aerodynamic trucks out there, but that makes the airflow the more interesting! 2x Hashtag#Intel Xeon 6979P 120-core CPUs computed+rendered this in ~28 hours.

Simulation Stats:
- FluidX3D CFD software: github.com/ProjectPhysX/FluidX3D
- Intel CPU Runtime for OpenCL (oneAPI)
- Lattice Boltzmann (LBM), D3Q19 SRT, FP32 arithmetic, FP32 memory storage
- 1389×2777×1111 = 4.3 Billion grid cells, 1 cell = (2.4 mm)³
- 380 GB memory footprint (55 Bytes/cell, or 19M cells per 1GB)
- 48750 time steps = 0.6 seconds real time
- 1800 4k images rendered, velocity-colored Q-criterion isosurfaces visualized
- 50 km/h airspeed
- Reynolds number = 1.9M
- Runtime = 1d03h58m (total) = 1d00h15m (LBM compute) + 3h43m (rendering)
- Average LBM performance = 2261 MLUPs/s

Hardware Specs:
- 2x Intel® Xeon® 6979P Prozessor (Granite Rapids), 2x 120 P-Cores, 2x 504MB Cache: ark.intel.com/content/www/us/en/ark/products/240781/intel-xeon-6979p-processor-504m-cache-2-10-ghz.html
- 24x 128GB 8800MT/s MRDIMMs, for 3TB total RAM at 1.7TB/s
- 0x #GPUs

Chevrolet C10 1961 in 4.3 Billion cell CFD - FluidX3D on 2x Intel Xeon 6979P

Running Cyberpunk 2077 in Windows CMD with 128x42 ASCII characters

There is a horrible bug in my code.

Smooth isosurface raytracing in real time

FluidX3D [FLUID SIMULATION] 65536 Particles in Real Time

Nuclide3D - 3D Table of Nuclides

FluidX3D [FLUID SIMULATION] Rocket Nozzle Simulation

Lightning fast CFD free and accessible for everyone - 2 years of FluidX3D on GitHub

The raindrop again but with 8 billion (!) cells in 512GB GPU memory

PhysX3D [GRAVITY SIMULATION] Ulysses slingshot on November 4th 2098

FluidX3D v2.14 Update Trailer

$Combined scientific CFD simulation and interactive raytracing with OpenCL (IWOCL & SYCLcon 2022) This is my IWOCL & SYCLcon 2022 technical talk. Extended Abstract: https://www.researchgate.net/publication/360501260_Combined_scientific_CFD_simulation_and_interactive_raytracing_with_OpenCL Slides: https://www.iwocl.org/wp-content/uploads/06-presentation-iwocl-syclcon-2022-lehmann.pdf 0:00 Intro & Demos 4:16 Basic Idea 6:54 Lattice Boltzmann CFD 8:35 Benchmarks & Efficiency 11:33 Application 12:37 Graphics with OpenCL 23:06 Conclusions 24:16 More Demos!! 26:09 Outro One of the main uses for OpenCL is compute applications where rendering is done externally. However separating simulation and rendering has many disadvantages, especially the slowdown caused by copying data from device to host, and needing to store raw data on the hard drive. A much faster approach is to implement both simulation and rendering in OpenCL. The rendering kernels have direct read-only access to the raw simulation data in ultra-fast GPU memory. This eliminates almost all PCIe data transfer, allowing for interactive visualization in real time while the simulation is running. Although OpenCL does not have existing functionality for graphical rendering, being a general compute language, it allows for implementing an entire graphics engine. On top, specific low-level optimizations make this OpenCL graphics engine outperform any existing rendering solution for this scenario, enabling fluid raytracing in real time on even non-RTX GPUs. This is demonstrated with the software FluidX3D [1]. First I introduce the lattice Boltzmann method for simulating physically accurate fluid flow and discuss optimizations: Being a memory-bound algorithm, coalesced memory access is key. This is achieved through AoS data layout and the one-step-pull LBM streaming scheme. One-step-pull leverages that misaligned read penalty is much smaller than misaligned write penalty on most GPUs. Roofline analysis shows that with these optimizations, the LBM runs at 100% efficiency on the fastest GPUs [2]. For free surface flows, the LBM is extended with an efficient Volume-of-Fluid (VoF) implementation [3]. This allows covering new grounds in science: FluidX3D has been used to simulate more than 1600 raindrop impacts to evaluate how microplastics transition from the ocean surface into the atmosphere [4]. Rasterization on the GPU is parallelized not over pixels but lines/triangles instead. Each line/triangle is transformed from 3D to 2D screen coordinates and then rasterized onto the frame (integer array) with Bresenham [5] and z-buffer. The raytracing graphics are based on a combination of fast ray-grid traversal and marching-cubes, leveraging that the computational grid from the LBM already is an ideal acceleration structure for raytracing. The idea is simple: Through each pixel on screen, shoot a reverse light ray out of the camera and see where it intersects with a surface in the scene. Then (recursively) calculate reflected/refracted rays and mix colors. If a ray doesnt intersect anything, its color is determined by the skybox. With mesh surfaces consisting of triangles, computation time quickly becomes a problem, as all triangles have to be tested for intersection. To overcome this, an acceleration structure is required. While games often use a bounding volume hierarchy, the LBM already provides an ideal alternative acceleration structure: the simulation grid. The corresponding algorithm is called ray-grid traversal: When a ray shoots through the 3D grid, intersections with the surface only have to be checked for at each traversed grid cell rather than the entire grid. Triangles are generated on-the-fly with marching-cubes and ray-triangle intersections are checked with Möller-Trumbore. If an intersection has been found, the normals are calculated on the 8 grid points spanning the cell, and are trilinearly interpolated to the intersection. The interpolated surface normal makes the raytraced surface appear smooth. On the GPU, the ray(s) for each pixel are computed in parallel. It is important how to align the OpenCL workgroups: best performance is achieved for 8x8 pixel tiles; this is about 50% faster than 64x1 tiles, because all rays of the workgroup are more likely to traverse the same grid cells, improving memory broadcasting. For marching-cubes, the algorithm by Paul Bourke [6] is implemented in OpenCL. Table size is reduced to 1/8. The Möller-Trumbore algorithm [7] is implemented branchless. This raytracing implementation is fast enough to run in real time for even the largest lattice dimensions that fit into GPU memory. This is demonstrated on the most realistic simulation of an impacting raindrop ever done [8]. [1] https://doi.org/10.15495/EPub_UBT_00005400 [2] https://arxiv.org/abs/2112.08926 [3] https://doi.org/10.3390/computation10020021 [4] https://doi.org/10.1186/s43591-021-00018-8 [5] https://doi.org/10.1147/sj.41.0025 [6] http://paulbourke.net/geometry/polygonise/ [7] https://doi.org/10.1080/10867651.1997.10487468 [8] https://youtu.be/1g_zFsvScME$

Dr. Moritz Lehmann |

Chevrolet C10 1961 in 4.3 Billion cell CFD - FluidX3D on 2x Intel Xeon 6979P