Uploaded October 2024 | Updated September 2026, 2 weeks ago
At Ray Summit 2024, Venkata Jagannath from Astronomer and Marwan Sarieddine from Anyscale present a comprehensive solution for AI infrastructure orchestration. The talk addresses the common challenge organizations face in creating a cohesive AI infrastructure, often resulting in disconnected platforms for data processing, model training, and inference.
The speakers demonstrate how the integration of Apache Airflow and Ray creates a unified system capable of handling all aspects of AI development and deployment, regardless of data or model size. They showcase an end-to-end ML lifecycle implementation for custom data, covering automated LLM fine-tuning, evaluation, serving, and LoRA deployments. This session offers practical insights for organizations looking to streamline their AI operations, reduce development time, and optimize costs in production environments.
--
Interested in more?
- Watch the full Day 1 Keynote: youtu.be/jwZHJthQvXo
- Watch the full Day 2 Keynote youtu.be/Lury2ad6KG8
--
đź”— Connect with us:
- Subscribe to our YouTube channel: youtube.com/@anyscale
- Twitter: https://x.com/anyscalecompute
- LinkedIn: linkedin.com/company/joinanyscale
- Website: anyscale.com
At Ray Summit 2024, Venkata Jagannath from Astronomer and Marwan Sarieddine from Anyscale present a comprehensive solution for AI infrastructure orchestration. The talk addresses the common challenge organizations face in creating a cohesive AI infrastructure, often resulting in disconnected platforms for data processing, model training, and inference.
The speakers demonstrate how the integration of Apache Airflow and Ray creates a unified system capable of handling all aspects of AI development and deployment, regardless of data or model size. They showcase an end-to-end ML lifecycle implementation for custom data, covering automated LLM fine-tuning, evaluation, serving, and LoRA deployments. This session offers practical insights for organizations looking to streamline their AI operations, reduce development time, and optimize costs in production environments.
--
Interested in more?
- Watch the full Day 1 Keynote: youtu.be/jwZHJthQvXo
- Watch the full Day 2 Keynote youtu.be/Lury2ad6KG8
--
đź”— Connect with us:
- Subscribe to our YouTube channel: youtube.com/@anyscale
- Twitter: https://x.com/anyscalecompute
- LinkedIn: linkedin.com/company/joinanyscale
- Website: anyscale.com
