Uploaded October 2024 | Updated September 2026, 1 week ago

---

Dean Wampler from IBM presents how Ray is being utilized for large-scale data processing in AI and scientific research. The session focuses on the Data Prep Kit, an open-source project developed by IBM Research and the AI Alliance, which uses Ray as its core driver for data processing tasks crucial to LLM training and tuning.

The presentation demonstrates Ray's capability to enable easy and resource-efficient scaling from laptop environments to massive-scale parallel processing of terabyte-sized datasets. Wampler also discusses Ray's role in the AI Alliance's Open Trusted Data initiative, particularly in data curation.

--

Interested in more?
- Watch the full Day 1 Keynote: youtu.be/jwZHJthQvXo
- Watch the full Day 2 Keynote youtu.be/Lury2ad6KG8
- Check out the Ray Summmit Breakout sessions youtube.com/playlist?list=PLzTswPQNepXntmT8jr9WaNfqQ60QwW7-U&si=qPw-_SxT9lVmbRGE

--

🔗 Connect with us:
- Subscribe to our YouTube channel: youtube.com/@anyscale
- Twitter: https://x.com/anyscalecompute
- LinkedIn: linkedin.com/company/joinanyscale
- Website: anyscale.com