From Model Serving To Distributed Inference: How Llm-d Evolves AI Platforms on… K. Yan & L. He @cncf

Uploaded August 2026 | Updated September 2026, 2 weeks ago

Don't miss out! Join us at our next KubeCon + CloudNativeCon events in Shanghai, China (8-9 September, 2026) and Salt Lake City, United States (Nov 9–12, 2026). Connect with our current graduated, incubating, and sandbox projects as the community gathers to further the education and advancement of cloud native computing. Learn more at kubecon.io

From Model Serving To Distributed Inference: How Llm-d Evolves AI Platforms on Kubernetes - Kay Yan, DaoCloud & Linbo He, Microsoft

This session explores how platform teams can evolve from basic model serving to production-grade distributed inference on Kubernetes. KAITO streamlines model onboarding and lifecycle management with curated presets, GPU node auto-provisioning, autoscaling, OpenAI-compatible endpoints, and support for runtimes such as vLLM. llm-d extends this foundation with inference-aware scheduling, KV-cache-aware routing, cross-node cache coordination, prefill/decode disaggregation, and workload-aware autoscaling. Rather than treating deployment and inference routing as separate concerns, this talk presents a practical reference architecture that brings both together on Kubernetes. It also explains how Gateway API Inference Extension can serve as a common interface, including InferencePool, Endpoint Picker, and Body Based Routing, and when teams should move from basic serving to distributed inference.

From Model Serving To Distributed Inference: How Llm-d Evolves AI Platforms on… K. Yan & L. He

$Keycloak + Sigstore: Binding Human Identity To Artifact Signatures - Oshi Gupta & Sagar Utekar Dont miss out! Join us at our next KubeCon + CloudNativeCon events in Shanghai, China (8-9 September, 2026) and Salt Lake City, United States (Nov 9–12, 2026). Connect with our current graduated, incubating, and sandbox projects as the community gathers to further the education and advancement of cloud native computing. Learn more at https://kubecon.io Keycloak + Sigstore: Binding Human Identity To Artifact Signatures - Oshi Gupta, Improving Pune (Infracloud Technologies) & Sagar Utekar, Crowdstrike Most teams treat signing and identity as two separate problems. Sigstore handles your artifact signatures. Keycloak handles your users. Nobody asks whether the person who triggered that signed build is actually who the signature claims they are. That gap matters more than people realize. In this talk we will walk through how we wired Keycloak directly into Sigstores keyless signing flow as the OIDC provider — so every artifact signature is cryptographically bound to a verified human or service identity that lives in your own identity infrastructure, not GitHubs, not Googles. Your keys, your trust root, your audit trail. Well cover the exact Fulcio configuration that makes this work, how Keycloak realm and client setup maps to Sigstores identity claims, and the operational realities nobody documents — token expiry during long builds, claim mapping mismatches that silently break verification, and how to handle service identities for automated pipelines alongside human developer identities in the same trust domain. If you care about knowing not just what was signed but who actually signed it and whether that person had the right to — this talk is for you.$

A Decade of Cilium Around the World - Liz Rice, Isovalent at Cisco & Hiroki Hanada, Cybozu, Inc.

Argo CD Maintainers Panel - Dan Garfield, Joanna Wyganowska, Michael Crenshaw & Nitish Kumar

Vulnerability Response for Large Open Source Projects - Jo Guerreiro & Charline Voinot, Grafana Labs

Simon Forster: Why Non-Code Contributions Matter | KubeCon Japan

Running OpenSearch at Scale in High-Traffic Gaming Systems - Siddharth Vijay, Baazi Games

Why CNCF Ambassador Sharma Shivlal Keeps Coming Back

OpenTelemetry Celebrates Graduation and the Next Era of Agentic... - Alolita Sharma & Ted Young

Project Lightning Talk: Youki: Whats New and Whats Next? - Yuta Nagai, CyberAgent Inc.

Turning Platform Engineering Work Into Business Value Leadership Understands - D. Cook & S. Forster