Uploaded October 2026 | Updated November 2026, 2 days ago
Pulling large model weights over public networks on every cold start wastes minutes you do not have.
Evan Johnson and Aveek Saha walk through a production inference setup on Linode Kubernetes Engine (LKE) that fixes that bottleneck. They wire up OpenWebUI, KubeRay, and Ray Serve on NVIDIA Blackwell GPUs, carve up hardware with MIG to run 16 parallel OCR workers, and drop cold starts from minutes to 43 seconds using local object storage caching.
### π Useful Links & Resources:
* Akamai Developers on GitHub: github.com/akamai-developers
* Akamai Developer Hub: developers.akamai.com
* $100 Akamai Cloud Credit: login.linode.com/signup?promo=akm-lp-akmtube-100-111325-T853
### π² Follow Us & Join Our Community
π X: https://x.com/akamaidevs?s=20
πΌ LinkedIn: linkedin.com/showcase/akamai-dev
πΎ Discord: discord.gg/5MJegzS8Y
### β±οΈ Timestamps:
0:00 Zero to AI Inference Overview
0:19 Reference Architecture: LKE, KubeRay, and Gateway API
0:43 Provisioning Infrastructure with Terraform and Tilt
1:25 Agentic Research Pipeline with MiniMax and Nemotron
1:51 Model Caching with Akamai Object Storage
2:27 Splitting GPUs with MIG Partitions for OCR
3:20 FP4 Quantization on NVIDIA Blackwell GPUs
3:51 Akamai Cloud Node Pools and GPU Allocation
4:42 Model Cache Startup Performance in Grafana
5:44 Running the Deep Research Agent in OpenWebUI
6:12 Real-Time GPU Metrics: Power, Temp, and Utilization
6:43 Ray Serve and vLLM Latency Dashboards
7:46 Research Agent Output and Key Findings
8:02 GitHub Repo Walkthrough and Wrap Up
Pulling large model weights over public networks on every cold start wastes minutes you do not have.
Evan Johnson and Aveek Saha walk through a production inference setup on Linode Kubernetes Engine (LKE) that fixes that bottleneck. They wire up OpenWebUI, KubeRay, and Ray Serve on NVIDIA Blackwell GPUs, carve up hardware with MIG to run 16 parallel OCR workers, and drop cold starts from minutes to 43 seconds using local object storage caching.
### π Useful Links & Resources:
* Akamai Developers on GitHub: github.com/akamai-developers
* Akamai Developer Hub: developers.akamai.com
* $100 Akamai Cloud Credit: login.linode.com/signup?promo=akm-lp-akmtube-100-111325-T853
### π² Follow Us & Join Our Community
π X: https://x.com/akamaidevs?s=20
πΌ LinkedIn: linkedin.com/showcase/akamai-dev
πΎ Discord: discord.gg/5MJegzS8Y
### β±οΈ Timestamps:
0:00 Zero to AI Inference Overview
0:19 Reference Architecture: LKE, KubeRay, and Gateway API
0:43 Provisioning Infrastructure with Terraform and Tilt
1:25 Agentic Research Pipeline with MiniMax and Nemotron
1:51 Model Caching with Akamai Object Storage
2:27 Splitting GPUs with MIG Partitions for OCR
3:20 FP4 Quantization on NVIDIA Blackwell GPUs
3:51 Akamai Cloud Node Pools and GPU Allocation
4:42 Model Cache Startup Performance in Grafana
5:44 Running the Deep Research Agent in OpenWebUI
6:12 Real-Time GPU Metrics: Power, Temp, and Utilization
6:43 Ray Serve and vLLM Latency Dashboards
7:46 Research Agent Output and Key Findings
8:02 GitHub Repo Walkthrough and Wrap Up




