Uploaded October 2024 | Updated September 2026, 2 weeks ago
As machine learning models grow in complexity and power, large-scale batch inference has become a critical challenge for enterprises. In this session, Richard Liaw and Scott Lee from Anyscale present their innovative approach to optimizing batch inference using Ray Data and Anyscale's infrastructure stack.
The speakers delve into the limitations of legacy systems and online serving endpoints for handling large-scale, latency-insensitive workloads. They showcase Anyscale's proprietary advancements in Ray Data, infrastructure design, and model optimization, demonstrating how these innovations combine to create a batch inference stack that significantly outperforms alternatives in cost-efficiency. This talk offers valuable insights for organizations looking to enhance their batch inference capabilities and reduce operational costs in machine learning deployments.
--
Interested in more?
- Watch the full Day 1 Keynote: youtu.be/jwZHJthQvXo
- Watch the full Day 2 Keynote youtu.be/Lury2ad6KG8
--
đź”— Connect with us:
- Subscribe to our YouTube channel: youtube.com/@anyscale
- Twitter: https://x.com/anyscalecompute
- LinkedIn: linkedin.com/company/joinanyscale
- Website: anyscale.com
As machine learning models grow in complexity and power, large-scale batch inference has become a critical challenge for enterprises. In this session, Richard Liaw and Scott Lee from Anyscale present their innovative approach to optimizing batch inference using Ray Data and Anyscale's infrastructure stack.
The speakers delve into the limitations of legacy systems and online serving endpoints for handling large-scale, latency-insensitive workloads. They showcase Anyscale's proprietary advancements in Ray Data, infrastructure design, and model optimization, demonstrating how these innovations combine to create a batch inference stack that significantly outperforms alternatives in cost-efficiency. This talk offers valuable insights for organizations looking to enhance their batch inference capabilities and reduce operational costs in machine learning deployments.
--
Interested in more?
- Watch the full Day 1 Keynote: youtu.be/jwZHJthQvXo
- Watch the full Day 2 Keynote youtu.be/Lury2ad6KG8
--
đź”— Connect with us:
- Subscribe to our YouTube channel: youtube.com/@anyscale
- Twitter: https://x.com/anyscalecompute
- LinkedIn: linkedin.com/company/joinanyscale
- Website: anyscale.com
