Networking in the Age of Inference with Hedgehog @Techfieldday
Networking in the Age of Inference with Hedgehog  @Techfieldday
Uploaded October 2026 | Updated November 2026, 6 days ago
Hedgehog explained how the shift from AI training to inference is reshaping the data center network. Marc Austin, Manish Vachharajani, and Sergei Lukianov covered disaggregated inference, which splits prefill and decode across different accelerators, the role of KV cache transfers over a scale-out RDMA Ethernet fabric, and multi-plane back-end designs that keep online inference clusters available. The session took place at Networking Field Day 41, with Austin, Hedgehog’s founder and CEO, setting up the business case before handing off to Vachharajani, the company’s CTO.

Austin framed the session around goodput, meaning time to first token and token throughput, and pointed to a Computex demonstration by customer SambaNova in which a sample workload took 90 seconds on an NVIDIA B200 alone but 30 seconds when paired with an air-cooled SambaNova SN40 RDU over a Hedgehog AI network. Vachharajani explained why: prefill is compute-bound, decode is limited by memory bandwidth, and a single model-in-a-box layout cannot optimize for both. Splitting them across separate pools, however, means shipping the KV cache from prefill to decode. At roughly 320 KB per token for Llama 3, a 32K-token prompt produces about 10 GB, and that transfer sits directly on the user’s response path, taking about 860 milliseconds at 100 Gbps, 215 milliseconds at 400 Gbps, and 100 milliseconds at 800 Gbps per GPU.

The result, Vachharajani argued, is that inference now needs the full scale-out RDMA back-end network once reserved for training, plus a KV cache storage tier, and unlike training it must stay online around the clock. Faster B300 NICs with 800 Gbps breakouts push designs toward multi-plane fabrics, two disconnected Clos networks that libraries such as UCX and NIXL can route across, so an entire plane can be taken offline for maintenance at reduced performance. Hedgehog is designing a successor cluster that adds a converged tier connecting both planes to storage and accelerators that cannot handle multi-plane networking, a topology outside standard reference architectures. Because pairing GPUs with decode accelerators from SambaNova, Groq, or Cerebras requires an interoperable interconnect, Vachharajani said that interconnect today is Ethernet with RoCE, which puts the network at the center of disaggregated inference.

Presented by Marc Austin, CEO and Founder, and Manish Vachharajani, CTO. Recorded live at Networking Field Day 41 in Santa Clara, CA on October 8, 2026. Watch the entire presentation at techfieldday.com/appearance/hedgehog-presents-at-networking-field-day-41 or visit techfieldday.com/event/nfd41 or GitHedgehog.com for more information.
Networking in the Age of Inference with HedgehogCommvault AI Infrastructure: Navigating Nonlinear Data Paths & Agentic SecuritySecure the Workforce with CiscoArrcus Unified AI Networking Fabric: One Programmable Fabric from Edge to CloudThe 4+1 AI Infrastructure Model with Keith TownsendUnlock Stranded Capacity & Tier Zero NVMe with HammerspaceCoreWeave - The Essential Cloud for AlCTERA: Why 2026 Belongs to Code-Executing AI AgentsInfrastructure as Code vs. Backup: Dreeze Made Strategic ChoicesConnecting the AI Loop with CoreWeave ForgeReimagining Security and Trust for the Agentic Era with CiscoCoder Presents at Tech Field Day at KubeCon North America 2026 #TFDx #KubeCon #TFDLive
Tech Field Day |

Networking in the Age of Inference with Hedgehog

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER