Uploaded October 2024 | Updated September 2026, 2 weeks ago
Rubrik's quest for high-performance, real-time AI inference led them to a game-changing solution: Ray Serve. In this technical deep dive, Shaikh Ismail and Shivanshu Agrawal from Rubrik unveil their journey of harnessing Ray's ML model serving library to meet demanding scalability and throughput requirements.
The duo explores Ray Serve's unique capabilities that made it stand out among alternatives for online inference scenarios. They share insights on how Rubrik successfully handles millions of evaluations daily, addressing critical challenges such as fault tolerance, robustness, and seamless deployment on Kubernetes. This session offers valuable lessons for organizations looking to elevate their AI serving infrastructure to meet the demands of high-stakes, real-time applications.
--
Interested in more?
- Watch the full Day 1 Keynote: youtu.be/jwZHJthQvXo
- Watch the full Day 2 Keynote youtu.be/Lury2ad6KG8
- Check out the Ray Summmit Breakout sessions youtube.com/playlist?list=PLzTswPQNepXntmT8jr9WaNfqQ60QwW7-U&si=qPw-_SxT9lVmbRGE
--
đź”— Connect with us:
- Subscribe to our YouTube channel: youtube.com/@anyscale
- Twitter: https://x.com/anyscalecompute
- LinkedIn: linkedin.com/company/joinanyscale
- Website: anyscale.com
Rubrik's quest for high-performance, real-time AI inference led them to a game-changing solution: Ray Serve. In this technical deep dive, Shaikh Ismail and Shivanshu Agrawal from Rubrik unveil their journey of harnessing Ray's ML model serving library to meet demanding scalability and throughput requirements.
The duo explores Ray Serve's unique capabilities that made it stand out among alternatives for online inference scenarios. They share insights on how Rubrik successfully handles millions of evaluations daily, addressing critical challenges such as fault tolerance, robustness, and seamless deployment on Kubernetes. This session offers valuable lessons for organizations looking to elevate their AI serving infrastructure to meet the demands of high-stakes, real-time applications.
--
Interested in more?
- Watch the full Day 1 Keynote: youtu.be/jwZHJthQvXo
- Watch the full Day 2 Keynote youtu.be/Lury2ad6KG8
- Check out the Ray Summmit Breakout sessions youtube.com/playlist?list=PLzTswPQNepXntmT8jr9WaNfqQ60QwW7-U&si=qPw-_SxT9lVmbRGE
--
đź”— Connect with us:
- Subscribe to our YouTube channel: youtube.com/@anyscale
- Twitter: https://x.com/anyscalecompute
- LinkedIn: linkedin.com/company/joinanyscale
- Website: anyscale.com
