Uploaded August 2026 | Updated September 2026, 2 weeks ago
Don't miss out! Join us at our next KubeCon + CloudNativeCon events in Shanghai, China (8-9 September, 2026) and Salt Lake City, United States (Nov 9–12, 2026). Connect with our current graduated, incubating, and sandbox projects as the community gathers to further the education and advancement of cloud native computing. Learn more at kubecon.io
Conformance for Inference: How We Reduced Bad Deploys on a GPU Platform - Aditya Soni, SailPoint & Hrittik Roy, vCluster
Inference on GPUs fails in repetitive ways: wrong image or artifact, mismatched CUDA or runtime, undersized GPU memory, bad resource requests, or a model that passes offline checks but regresses under real traffic. On a shared k8s GPU platform, those mistakes become multi-tenant incidents - noisy neighbors, OOMKills, SLO breaches, and rollbacks that waste accelerator time.
This talk describes how one team built conformance for inference workloads, checks applied before production traffic, covering container and model artifacts, GPU capacity and visibility contracts, health and readiness semantics, and minimum observability (metrics/traces where used).
Attendees leave with a practical checklist they can reuse:
- How to separate “builds” from “serving” conformance,
- How to catch regressions early, and
- How to align GPU scheduling and quotas with inference SLOs.
We will share what worked, what did not, what teams pushed back on, and a short checklist for platform and app owners.
Don't miss out! Join us at our next KubeCon + CloudNativeCon events in Shanghai, China (8-9 September, 2026) and Salt Lake City, United States (Nov 9–12, 2026). Connect with our current graduated, incubating, and sandbox projects as the community gathers to further the education and advancement of cloud native computing. Learn more at kubecon.io
Conformance for Inference: How We Reduced Bad Deploys on a GPU Platform - Aditya Soni, SailPoint & Hrittik Roy, vCluster
Inference on GPUs fails in repetitive ways: wrong image or artifact, mismatched CUDA or runtime, undersized GPU memory, bad resource requests, or a model that passes offline checks but regresses under real traffic. On a shared k8s GPU platform, those mistakes become multi-tenant incidents - noisy neighbors, OOMKills, SLO breaches, and rollbacks that waste accelerator time.
This talk describes how one team built conformance for inference workloads, checks applied before production traffic, covering container and model artifacts, GPU capacity and visibility contracts, health and readiness semantics, and minimum observability (metrics/traces where used).
Attendees leave with a practical checklist they can reuse:
- How to separate “builds” from “serving” conformance,
- How to catch regressions early, and
- How to align GPU scheduling and quotas with inference SLOs.
We will share what worked, what did not, what teams pushed back on, and a short checklist for platform and app owners.
