Uploaded October 2024 | Updated September 2026, 2 weeks ago
As machine learning becomes integral to business operations, data engineering teams often find themselves at the forefront of ML infrastructure development. Sam Hallam from Reverb shares the company's journey in this transformative process, offering valuable insights for organizations navigating similar transitions.
Hallam delves into the challenges and learnings encountered while building a scalable, developer-friendly ML platform, with a particular focus on leveraging Ray and Anyscale. The talk covers crucial aspects such as crafting an exceptional developer experience, streamlining model deployment, and implementing effective cost and performance monitoring. This session provides a roadmap for data teams looking to expand their capabilities into the realm of MLOps, offering practical strategies for building robust ML infrastructure.
--
Interested in more?
- Watch the full Day 1 Keynote: youtu.be/jwZHJthQvXo
- Watch the full Day 2 Keynote youtu.be/Lury2ad6KG8
- Check out the Ray Summmit Breakout sessions youtube.com/playlist?list=PLzTswPQNepXntmT8jr9WaNfqQ60QwW7-U&si=qPw-_SxT9lVmbRGE
--
đź”— Connect with us:
- Subscribe to our YouTube channel: youtube.com/@anyscale
- Twitter: https://x.com/anyscalecompute
- LinkedIn: linkedin.com/company/joinanyscale
- Website: anyscale.com
As machine learning becomes integral to business operations, data engineering teams often find themselves at the forefront of ML infrastructure development. Sam Hallam from Reverb shares the company's journey in this transformative process, offering valuable insights for organizations navigating similar transitions.
Hallam delves into the challenges and learnings encountered while building a scalable, developer-friendly ML platform, with a particular focus on leveraging Ray and Anyscale. The talk covers crucial aspects such as crafting an exceptional developer experience, streamlining model deployment, and implementing effective cost and performance monitoring. This session provides a roadmap for data teams looking to expand their capabilities into the realm of MLOps, offering practical strategies for building robust ML infrastructure.
--
Interested in more?
- Watch the full Day 1 Keynote: youtu.be/jwZHJthQvXo
- Watch the full Day 2 Keynote youtu.be/Lury2ad6KG8
- Check out the Ray Summmit Breakout sessions youtube.com/playlist?list=PLzTswPQNepXntmT8jr9WaNfqQ60QwW7-U&si=qPw-_SxT9lVmbRGE
--
đź”— Connect with us:
- Subscribe to our YouTube channel: youtube.com/@anyscale
- Twitter: https://x.com/anyscalecompute
- LinkedIn: linkedin.com/company/joinanyscale
- Website: anyscale.com
![[Ray Meetup] Ray + vLLM in Action: Lessons from Pinterest and Large Scale Distributed Inference](https://i.ytimg.com/vi/HDSy09hrm2I/mqdefault.jpg "[Ray Meetup] Ray + vLLM in Action: Lessons from Pinterest and Large Scale Distributed Inference
Listen in to our Ray Meetup where we explored batch inference at scale with Ray and vLLM! Learn how Pinterest scales batch inference using Ray, and get a first look at Anyscale’s latest tools—Ray Serve and Data LLM—for orchestrating large-scale LLM inference. We’ll cover topics like batch inference, prefill-decode disaggregation, DP/EP parallelism, and custom request routing.
Speakers:
Chia-Wei Chen, Software Engineer, ML Training Infra, Pinterest
Kourosh Hakhamaneshi, AI Lead, Anyscale")
