Uploaded April 2026 | Updated September 2026, 3 weeks ago
Low Latency Serving of Offline Data: Efficient, Safe, and Reliable Data Loading At Scale
William Schor
Transferring massive datasets (e.g., 50 TB) into online serving systems—without disrupting live applications—poses unique challenges. Traditional methods often lead to high costs, slow transfers, and performance bottlenecks. In this talk, we present a novel system that preprocesses data offline into RocksDB SST files, stages them in the cloud, and loads them into isolated online infrastructure for seamless, near-instantaneous cutovers. Our approach enables robust validation, rapid rollback, and high-scale, low-latency reads, all while dramatically reducing deployment time (by 99%) and cost (by 70%). Attendees will learn about the architectural innovations, technical lessons, and practical strategies that make this system a game-changer for large-scale data operations.
View the full SREcon26 Americas program at usenix.org/conference/srecon26americas/program
Low Latency Serving of Offline Data: Efficient, Safe, and Reliable Data Loading At Scale
William Schor
Transferring massive datasets (e.g., 50 TB) into online serving systems—without disrupting live applications—poses unique challenges. Traditional methods often lead to high costs, slow transfers, and performance bottlenecks. In this talk, we present a novel system that preprocesses data offline into RocksDB SST files, stages them in the cloud, and loads them into isolated online infrastructure for seamless, near-instantaneous cutovers. Our approach enables robust validation, rapid rollback, and high-scale, low-latency reads, all while dramatically reducing deployment time (by 99%) and cost (by 70%). Attendees will learn about the architectural innovations, technical lessons, and practical strategies that make this system a game-changer for large-scale data operations.
View the full SREcon26 Americas program at usenix.org/conference/srecon26americas/program
