Fixing S3 Bottlenecks: Scalable I/O for Ray with Alluxio | Ray Summit 2025 @anyscale
Fixing S3 Bottlenecks: Scalable I/O for Ray with Alluxio | Ray Summit 2025  @anyscale
Uploaded December 2025 | Updated November 2026, 50 minutes ago
At Ray Summit 2025, Bin Fan from Alluxio shares how a fast-moving AI startup transformed a fragile, high-throughput training pipeline into a production-grade data infrastructure—using Ray and Alluxio to accelerate model iteration for advanced document understanding and search systems.

He begins by highlighting the core challenge: rapid experimentation is crucial for startups, yet large-scale preprocessing and training pipelines often collapse under heavy I/O pressure. The team evaluated three architectural designs for integrating Ray with storage systems—each addressing different bottlenecks and scaling constraints.

1. Direct Ray → S3 Access (Initial Architecture)

The pipeline originally relied on Ray preprocessing workers writing directly to S3 while PyTorch handled training reads. Under real-world concurrency—1,000+ Ray workers writing simultaneously—the system suffered from:

Severe throughput degradation

High S3 egress and PUT/GET costs

Training slowdowns from inconsistent read performance

This setup proved unsustainable for fast model iteration.

2. Alluxio with Async Write-Back (Intermediate Architecture)

Adding Alluxio as an asynchronous write buffer improved read performance during training, but new bottlenecks surfaced:

Metadata saturation during peak concurrency

Instability in the write path

Frequent cluster crashes and job restarts

Throughput improved, but reliability did not.

3. Decentralized Alluxio 3.x with Cache-Through Writes (Final Architecture)

A fully decentralized Alluxio 3.x deployment with cache-through writes unlocked the performance and stability required:

Consistent ingestion under 400 Gbps internal bandwidth

Throttled 10 Gbps outbound writes to S3 for cost control

Zero restarts—even under bursty load

Significantly improved GPU utilization and end-to-end iteration speed

This architecture enabled fast, stable, scalable model development without expensive storage overhauls.

Liked this video? Check out other Ray Summit breakout session recordings youtube.com/playlist?list=PLzTswPQNepXllnU0C36WtkC0dqkAoDulh

Subscribe to our YouTube channel to stay up-to-date on the future of AI! youtube.com/c/anyscale

🔗 Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
Fixing S3 Bottlenecks: Scalable I/O for Ray with Alluxio | Ray Summit 2025Scaling Post-Training Workflows with Ray Data, Ray Data LLM, and vLLM | Ray Summit 2025Live Virtual Hands On Lab: Distributed Training at Scale with Ray and PyTorchA Quick Overview of the Ray Libraries Built on Ray Core | Ray Summit ExpoWhat is Anyscale in 8 minScaling LLMs at Apple: Ray Serve + vLLM Deep Dive | Ray Summit 2025Scaling Robot Policy Evaluations to Thousands of Parallel SimulationsWhat Makes RL the Hardest Workload in AI | Robert Nishihara (Anyscale) | Ray Summit 2026Ray Data: Whats New - GPU Shuffle, Unity Catalog, DataSourceV2 | Anyscale | Ray Summit 2026Inside Adobe Firefly: JIT-Embedding with Ray Serve for Faster GenAI Training | Ray Summit 2025Running Embedding Processing for AI Search at Scale | Notion + AnyscaleServing vLLM on Agentic Production Workloads | Inferact | Ray Summit 2026
Anyscale |

Fixing S3 Bottlenecks: Scalable I/O for Ray with Alluxio | Ray Summit 2025

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER