How Ray Data Powers Scalable AI Workloads | Ray Summit 2025 @anyscale

Uploaded November 2025 | Updated September 2026, 2 weeks ago

Slides: drive.google.com/file/d/1G3DPYUd9i5dxsGwI9QjAd7NJe0jMDah4/view?usp=sharing

At Ray Summit 2025, Balaji Veeramani from Anyscale shares how Ray Data has evolved into one of the most widely used libraries in the Ray ecosystem—purpose-built for the new generation of AI workloads.

Unlike traditional data processing engines, Ray Data is designed from the ground up for multimodal, accelerator-native, and AI-centric pipelines. In this talk, the speakers provide an overview of Ray Data’s core capabilities and highlight the major features added over the past year to support:

Large-scale batch inference across GPUs and clusters

Distributed training data preparation and ingestion for massive models

High-performance multimodal data processing, spanning images, video, text, and more

Whether you're building LLM pipelines, multimodal training workflows, or high-throughput inference systems, this session provides a clear look at how Ray Data powers modern AI at scale.

Liked this video? Check out other Ray Summit breakout session recordings youtube.com/playlist?list=PLzTswPQNepXllnU0C36WtkC0dqkAoDulh

Subscribe to our YouTube channel to stay up-to-date on the future of AI! youtube.com/c/anyscale

🔗 Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com

How Ray Data Powers Scalable AI Workloads | Ray Summit 2025

Inside Uber: Scaling Model Training with Ray | Ray Summit 2025

Transforming Multimodal Data Management with LanceDB-Ray | Ray Summit 2024

Why Ray Became a Distributed Computing Engine for Modern AI

RLlib: Lessons from the V2 Stack and Road Ahead | Ray Summit 2025

Pinterests ML Evolution: Distributed Training with Ray | Ray Summit 2024

How xAI Scales Image & Video Processing with Ray | Ray Summit 2025

Ion Stoica on Agentic Systems and AI Reliability | Ray on the Road – NYC 2025

Anyscale on Azure: Build and deploy AI at scale in your own tenant

An Overview of CloudKitchenss Ray-Powered ML Platform | Ray Summit 2024

$Prompt Learning: A Reinforcement Learning-Inspired Approach to AI Optimization | Ray Summit 2025 At Ray Summit 2025, Jason Lopatecki from Arize AI shares a new paradigm for iterative model improvement—Prompt Learning (PL)—a technique inspired by reinforcement learning but performed entirely through prompts, without weight updates, large datasets, or gradient-based training. He begins by framing the central question: What if we could achieve RL-like gains in model performance simply by refining prompts using natural-language feedback? Building on ideas from NVIDIA’s Voyager and broader community discussions, Arize explores how structured English feedback—evaluations, explanations, annotations—can act as an “error term” that guides models toward better behavior. Jason walks through the conceptual foundation of Prompt Learning and why encoding corrections directly in language can dramatically reduce data requirements—sometimes down to only a handful of examples. Key highlights include: The Prompt Learning framework: How PL adapts RL-style improvement loops to the prompt layer Why natural-language feedback is powerful: Turning textual “error signals” into prompt refinements without extra training PL vs RL efficiency: A head-to-head comparison showing where PL outperforms traditional RL-driven refinement Real-world experiment: Guiding JSON generation with latent constraints using only a few examples TextPRO system: Arize’s PL engine that performs end-to-end optimization through single LLM calls—at a fraction of the supervision cost Jason shares experimental results showing that Prompt Learning can achieve up to 70% test accuracy using just five feedback rules, outperforming standard prompting techniques and demonstrating that lightweight, language-based optimization can yield surprisingly strong results. Attendees will leave with a new lens on iterative LLM improvement—one that opens the door to powerful, low-cost prompt optimization without full RL pipelines or large training sets. Liked this video? Check out other Ray Summit breakout session recordings https://www.youtube.com/playlist?list=PLzTswPQNepXllnU0C36WtkC0dqkAoDulh Subscribe to our YouTube channel to stay up-to-date on the future of AI! https://www.youtube.com/c/anyscale 🔗 Connect with us: LinkedIn: https://www.linkedin.com/company/joinanyscale/$

Introducing Terminal-Bench: Evaluating LLM Agents in Realistic Terminal Settings | Ray Summit 2025