Serving GLM-5.3 Across Two NVIDIA DGX Stations with vLLM @NVIDIADeveloper
Serving GLM-5.3 Across Two NVIDIA DGX Stations with vLLM  @NVIDIADeveloper
Uploaded November 2026 | Updated November 2026, 9 hours ago
When a model is too large for one GPU, you can still run it locally. In this tutorial, we serve GLM-5.3, a 753-billion-parameter mixture-of-experts model, across two NVIDIA DGX Stations with vLLM and tensor parallelism, keeping every weight in GPU memory with no CPU offload.

The GLM-5.3 NVFP4 checkpoint is about 464 GB, more than the 252 GB of memory on a single NVIDIA GB300 GPU. By connecting two DGX Stations with ConnectX-8 and splitting the model across both GPUs, the two systems serve one model through a single OpenAI-compatible API.
You'll see how to prepare both DGX Stations and collect the network values for the RDMA connection, download the pinned checkpoint and copy it to the second Station, and launch the same vLLM script on both machines. We then verify the service: a health check on the API, an RDMA check that confirms the GPUs communicate directly over the network instead of falling back to TCP sockets, and nvidia-smi showing one vLLM worker on each GPU. Next, we send a chat request through the OpenAI-compatible endpoint and check the answer.

Finally, we use NVIDIA AIPerf with SPEED-Bench Coding prompts to measure how per-user throughput and request latency change as concurrency increases. The video runs a quick test at concurrency 1 and 8, then shows the reference results at 1, 8, 16, and 32 concurrent requests. The companion post on the NVIDIA Technical Blog covers the full benchmark and every command used in this walkthrough.

Running a frontier-scale model on your own hardware is useful for working with sensitive data, prototyping agents locally, and iterating on prompts without per-token costs. To get started, connect your two DGX Stations with the Connect Two DGX Stations playbook, then follow the step-by-step commands in the companion blog.


➑️ Learn more: Connect Two DGX Stations playbook: build.nvidia.com/station/connect-two-stations
πŸ“ Tech blog: developer.nvidia.com/blog/when-one-gpu-isnt-enough-serving-glm-5-3-across-two-nvidia-dgx-stations-with-vllm
πŸ“₯ Download: GLM-5.3 NVFP4 on Hugging Face: huggingface.co/nvidia/GLM-5.3-NVFP4

Chapters:
00:00 Introduction
00:16 Why one GPU isn't enough
00:55 Step 1: Prepare the two DGX Stations
01:40 Step 2: Download and copy the model
02:21 Step 3: Launch the distributed vLLM service
03:45 Step 4: Verify serving and send a request
05:01 Step 5: Benchmark concurrency with NVIDIA AIPerf
06:20 Recap
Serving GLM-5.3 Across Two NVIDIA DGX Stations with vLLMSeattle DGX Spark Hackathon Winners SpotlightJensen Huang on What Makes NVIDIAs Architecture UniqueNemotron 3 Ultra and the Open Model Landscape | Nemotron LabsSelf-Evolving Hermes Agents: Enterprise AI That Gets Better With Use | Nemotron LabsHow to Run Agent-Led Autoresearch with NVIDIA NeMo RL and NeMo GymWhy Open Data Matters | Nemotron LabsOpen-Model Indic Language AI with NVIDIA Nemotron 3.5 Super VLGenerate Synthetic Data for Physical AI With NVIDIA Brev Launchables and Agent SkillsSecuring Long-Running AI Agents: From Setup to SandboxingHow to Deploy a Standalone VLM Kiosk on NVIDIA Jetson with CodexStylized Stock Returns w/ KDE
NVIDIA Developer |

Serving GLM-5.3 Across Two NVIDIA DGX Stations with vLLM

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER