Uploaded June 2025 | Updated November 2026, 2 hours ago
A lack of appropriate data, decreased model performance, and other obstacles have made it difficult to expand the input language models can receive. Li Lyna Zhang introduces LongRoPE, a method capable of extending content windows to more than 2 million tokens.
Show notes: microsoft.com/en-us/research/podcast/abstracts-july-29-2024
Listen to the Abstracts series: microsoft.com/en-us/research/podcast-series/abstracts
A lack of appropriate data, decreased model performance, and other obstacles have made it difficult to expand the input language models can receive. Li Lyna Zhang introduces LongRoPE, a method capable of extending content windows to more than 2 million tokens.
Show notes: microsoft.com/en-us/research/podcast/abstracts-july-29-2024
Listen to the Abstracts series: microsoft.com/en-us/research/podcast-series/abstracts










