Uploaded July 2026 | Updated September 2026, 3 weeks ago
From adding GPUs a year before ChatGPT to building the cloud primitives behind elastic inference, agent sandboxes, post-training, and production AI workloads, Modal has quietly become one of the most important infrastructure companies in AI. In this episode, Modal CTO Akshat Bubna joins swyx and Vibhu after Modal’s Series C to unpack why AI applications don’t fit traditional cloud assumptions, why Kubernetes was never designed for bursty compute-heavy workloads, and why Modal is now shifting from developer experience to agent experience.
We go deep on Modal’s AI infra stack: serverless functions, decorator-based infrastructure, elastic inference for custom models, GPU snapshotting, DeFlash, speculative decoding, Auto Endpoints, sandboxes, persistent storage, networked containers, private IPv6, RDMA, multi-node training, and Modal’s capacity pool across 17 cloud providers. Akshat also explains why RL rollouts can require 100,000 sandboxes, why production agents need hard guardrails, why observability may matter more than reading code, and why AI has made infrastructure exciting again.
We discuss:
• Why Kubernetes wasn’t built for bursty AI workloads
• How Modal started as a better runtime before becoming an AI cloud
• Why Modal added GPUs a year before ChatGPT
• The shift from developer experience to agent experience
• Why observability matters when agents are writing the code
• Elastic inference for custom models across audio, video, robotics, and comp bio
• GPU snapshotting, cold starts, and why inference workloads are so bursty
• Why RL rollouts can require 100,000 sandboxes
• DeFlash, speculative decoding, and frontier-level inference performance
• Auto Endpoints and making optimized inference easier to deploy
• What Modal adds beyond vLLM, SGLang, and raw GPU rental
• Modal’s 17-cloud capacity pool and “supercloud” strategy
• Networked sandboxes, sidecars, private IPv6, and RDMA
• Serverless multi-node training for post-training and research workloads
• Auto-research, model-guided sweeps, and agents launching GPU experiments
• Compute strategy, capacity planning, and batch tiers
• Why production agents need specialized sandboxes and hard guardrails
• Modal’s take on managed agents, CI, Gitpod/Ona, Python, TypeScript, and Modal Bench
—
Akshat Bubna
• LinkedIn: linkedin.com/in/akshat-bubna-188885103
• X: https://x.com/akshat_b
Modal
• Website: modal.com
Timestamps
00:00:00 Hook
00:01:14 Introduction
00:01:53 Modal’s origin and why Kubernetes wasn’t enough
00:05:46 Developer Experience → Agent Experience
00:07:35 Modal’s AI cloud primitives
00:10:28 Sandboxes, agent loops, and proto-Cognition
00:13:26 Elastic inference, GPU snapshotting, and 100,000 sandboxes
00:16:38 DeFlash, speculative decoding, and Auto Endpoints
00:21:13 Production-grade inference beyond raw GPUs
00:23:14 Background agents, Ramp Inspect, and the agent lifecycle
00:25:22 Modal’s 17-cloud supercloud strategy
00:27:54 Networked sandboxes, private IPv6, and RDMA
00:34:02 Multi-node training, post-training, and auto research
00:38:50 Compute strategy, capacity planning, and batch tiers
00:42:09 Open models, real-time AI, and production agent infra
00:44:20 Hard guardrails, managed agents, and specialized sandboxes
00:47:20 Why AI made infrastructure exciting again
00:49:44 Model APIs, differentiated products, and agentic video
00:53:04 CI, coding-agent infra, SDKs, and Modal Bench
00:58:42 Closing Thoughts
From adding GPUs a year before ChatGPT to building the cloud primitives behind elastic inference, agent sandboxes, post-training, and production AI workloads, Modal has quietly become one of the most important infrastructure companies in AI. In this episode, Modal CTO Akshat Bubna joins swyx and Vibhu after Modal’s Series C to unpack why AI applications don’t fit traditional cloud assumptions, why Kubernetes was never designed for bursty compute-heavy workloads, and why Modal is now shifting from developer experience to agent experience.
We go deep on Modal’s AI infra stack: serverless functions, decorator-based infrastructure, elastic inference for custom models, GPU snapshotting, DeFlash, speculative decoding, Auto Endpoints, sandboxes, persistent storage, networked containers, private IPv6, RDMA, multi-node training, and Modal’s capacity pool across 17 cloud providers. Akshat also explains why RL rollouts can require 100,000 sandboxes, why production agents need hard guardrails, why observability may matter more than reading code, and why AI has made infrastructure exciting again.
We discuss:
• Why Kubernetes wasn’t built for bursty AI workloads
• How Modal started as a better runtime before becoming an AI cloud
• Why Modal added GPUs a year before ChatGPT
• The shift from developer experience to agent experience
• Why observability matters when agents are writing the code
• Elastic inference for custom models across audio, video, robotics, and comp bio
• GPU snapshotting, cold starts, and why inference workloads are so bursty
• Why RL rollouts can require 100,000 sandboxes
• DeFlash, speculative decoding, and frontier-level inference performance
• Auto Endpoints and making optimized inference easier to deploy
• What Modal adds beyond vLLM, SGLang, and raw GPU rental
• Modal’s 17-cloud capacity pool and “supercloud” strategy
• Networked sandboxes, sidecars, private IPv6, and RDMA
• Serverless multi-node training for post-training and research workloads
• Auto-research, model-guided sweeps, and agents launching GPU experiments
• Compute strategy, capacity planning, and batch tiers
• Why production agents need specialized sandboxes and hard guardrails
• Modal’s take on managed agents, CI, Gitpod/Ona, Python, TypeScript, and Modal Bench
—
Akshat Bubna
• LinkedIn: linkedin.com/in/akshat-bubna-188885103
• X: https://x.com/akshat_b
Modal
• Website: modal.com
Timestamps
00:00:00 Hook
00:01:14 Introduction
00:01:53 Modal’s origin and why Kubernetes wasn’t enough
00:05:46 Developer Experience → Agent Experience
00:07:35 Modal’s AI cloud primitives
00:10:28 Sandboxes, agent loops, and proto-Cognition
00:13:26 Elastic inference, GPU snapshotting, and 100,000 sandboxes
00:16:38 DeFlash, speculative decoding, and Auto Endpoints
00:21:13 Production-grade inference beyond raw GPUs
00:23:14 Background agents, Ramp Inspect, and the agent lifecycle
00:25:22 Modal’s 17-cloud supercloud strategy
00:27:54 Networked sandboxes, private IPv6, and RDMA
00:34:02 Multi-node training, post-training, and auto research
00:38:50 Compute strategy, capacity planning, and batch tiers
00:42:09 Open models, real-time AI, and production agent infra
00:44:20 Hard guardrails, managed agents, and specialized sandboxes
00:47:20 Why AI made infrastructure exciting again
00:49:44 Model APIs, differentiated products, and agentic video
00:53:04 CI, coding-agent infra, SDKs, and Modal Bench
00:58:42 Closing Thoughts
