Uploaded April 2024 | Updated September 2026, 2 weeks ago
Allen School Colloquia Series
Title: Scalable and Efficient Systems for Large Language Models
Speaker: Lianmin Zheng (Berkeley)
Date: Monday, April i, 2024
Abstract: Large Language Models (LLMs) have been driving recent breakthroughs in AI. These advancements would not have been possible without the support of scalable and efficient infrastructure systems. In this talk, I will introduce several systems I have designed and built to support the entire model lifecycle, from training to deployment to evaluation. First, I will present Alpa, a system for large-scale model-parallel training that automatically generates execution plans unifying inter- and intra-operator parallelism. Next, I will discuss efficient deployment systems, covering both the frontend programming interface and backend runtime optimizations for high-performance inference. Finally, I will complete the model lifecycle by presenting our model evaluation efforts, including the crowdsourced live benchmark platform, Chatbot Arena, and the automatic evaluation pipeline, LLM-as-a-Judge. These projects have collectively laid a solid foundation for large language model systems, being widely adopted by leading LLM developers and companies. I will conclude by outlining some future directions, such as a programmatic and composable software stack for using LLMs and further improvements with synthetic data.
Bio: Lianmin Zheng is a Ph.D. student in the EECS department at UC Berkeley, advised by Ion Stoica and Joseph E. Gonzalez. His research interests include machine learning systems, large language models, compilers, and distributed systems. He builds full-stack, scalable, and efficient systems to advance the development of AI. He co-founded LMSYS.org, where he leads impactful open-source large language model projects such as Vicuna and Chatbot Arena, which have received millions of downloads and served millions of users. He has received a Meta Ph.D. Fellowship, an IEEE Micro Best Paper Award, and an a16z open-source AI grant.
This video is closed captioned.
Allen School Colloquia Series
Title: Scalable and Efficient Systems for Large Language Models
Speaker: Lianmin Zheng (Berkeley)
Date: Monday, April i, 2024
Abstract: Large Language Models (LLMs) have been driving recent breakthroughs in AI. These advancements would not have been possible without the support of scalable and efficient infrastructure systems. In this talk, I will introduce several systems I have designed and built to support the entire model lifecycle, from training to deployment to evaluation. First, I will present Alpa, a system for large-scale model-parallel training that automatically generates execution plans unifying inter- and intra-operator parallelism. Next, I will discuss efficient deployment systems, covering both the frontend programming interface and backend runtime optimizations for high-performance inference. Finally, I will complete the model lifecycle by presenting our model evaluation efforts, including the crowdsourced live benchmark platform, Chatbot Arena, and the automatic evaluation pipeline, LLM-as-a-Judge. These projects have collectively laid a solid foundation for large language model systems, being widely adopted by leading LLM developers and companies. I will conclude by outlining some future directions, such as a programmatic and composable software stack for using LLMs and further improvements with synthetic data.
Bio: Lianmin Zheng is a Ph.D. student in the EECS department at UC Berkeley, advised by Ion Stoica and Joseph E. Gonzalez. His research interests include machine learning systems, large language models, compilers, and distributed systems. He builds full-stack, scalable, and efficient systems to advance the development of AI. He co-founded LMSYS.org, where he leads impactful open-source large language model projects such as Vicuna and Chatbot Arena, which have received millions of downloads and served millions of users. He has received a Meta Ph.D. Fellowship, an IEEE Micro Best Paper Award, and an a16z open-source AI grant.
This video is closed captioned.
![[Audio Descriptions] I Am CSE: Kyle Johnson](https://i.ytimg.com/vi/dYksN04tkCA/mqdefault.jpg "[Audio Descriptions] I Am CSE: Kyle Johnson
Allen School graduate student Kyle Johnson of the UW’s Mobile Intelligence Lab and Autonomous Insect Robotics Lab, shares his work on origami microfliers, tiny battery-free robots inspired by nature that can be used for environmental monitoring, as well as his role in A Vision for Engineering Leadership and Access (AVELA) focused on engaging underserved youth.
This video is closed captioned.
A version of this video without audio descriptions is available here: https://youtu.be/tyOcyn2LuIw.")
