SNIA SDC: StorageAI 2026 - From Heuristics to Principles: A Practice Model for LLM Inference @SNIAVideo
SNIA SDC: StorageAI 2026 - From Heuristics to Principles: A Practice Model for LLM Inference  @SNIAVideo
Uploaded May 2026 | Updated November 2026, 1 day ago
Large Language Model (LLM) inference at scale is governed by a complex interplay between compute, memory bandwidth, batching, and system architecture. Despite rapid innovation—such as Prefill-Decode Disaggregation (PDD), chunked prefill scheduling, and KV-cache offloading—practical deployment remains largely heuristic-driven, with limited understanding of when and why each technique delivers benefit.
This talk presents a simple but powerful analytical framework for reasoning about LLM inference performance in modern multi-GPU and multi-node systems. We introduce two core metrics: incremental prefill compute cost and incremental KV-cache memory load cost, capturing the fundamental trade-offs underlying inference efficiency. Using these building blocks, we derive closed-form models that explain the behavior of aggregated execution, PDD, and chunked prefill, both with and without KV-cache offloading.
The analysis reveals several non-obvious insights: when PDD provides real gains (and when it does not), how to optimally partition GPUs between prefill and decode, and why chunked prefill can achieve up to 2x throughput improvements by overlapping compute- and memory-bound phases. We further show how prefill acceleration through KV-cache reuse translates into end-to-end throughput gains under continuous batching and how it composes with other architectural techniques.
This framework provides actionable guidance for designing and operating high-performance, cost-efficient LLM inference systems - turning a complex optimization problem into a set of principled, predictable trade-offs.

Presented by Eshcar Hillel | AsteraLabs - Principal Research Scientist

Learn More:
• SDC: StorageAI Website: snia.org/sniadeveloper/storageai
• SNIA Website: snia.org
• SNIA Educational Library: snia.org/library
• X: twitter.com/SNIA
• LinkedIn: linkedin.com/company/snia
SNIA SDC: StorageAI 2026 - From Heuristics to Principles: A Practice Model for LLM InferenceSNIA SDC 2025  -Why Should We Have Yet Another Enterprise & Datacenter Standard Form Factor for SSDsSNIA Compute Memory Storage Community - 2026 SNIA PreviewSNIA SDC 2025  - Gen6 is coming, but what is needed from NV StorageWhat is Serial Attached SCSI?SNIA SDC 2025  - Discussion & Analysis of the MLPerf Storage Benchmark Suite & AI Storage WorkloadsPerformance-based TCO Model 2.0 WalkthroughSNIA SFF TWG - 2026 SNIA PreviewLiquid Cooling, SSD Design, And Standards For The AI EraSNIA Cloud Storage Technologies Community - 2026 SNIA PreviewSNIA Partner Update - TGG - June 2024SNIA Member Recognition Program - 2026 SNIA Preview
SNIAVideo |

SNIA SDC: StorageAI 2026 - From Heuristics to Principles: A Practice Model for LLM Inference

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER