Uploaded June 2026 | Updated September 2026, 3 weeks ago
Large language models process vast sequences of input tokens by alternating between classical multi-layer perceptron layers and self-attention mechanisms. While the approximation capabilities of perceptrons are relatively well understood, those of attention mechanisms remain less explored. In this talk, I will compare the proof techniques and approximation results associated with these two types of layers, emphasizing key open questions that connect large language models with approximation theory in infinite-dimensional spaces representing input token distributions.
Gabriel Peyré (CNRS, DMA, École Normale Supérieure)
===
Find this and many more scientific videos on carmin.tv - a French video platform for mathematics and their interactions with other sciences offering extra functionalities tailored to meet the needs of the research community.
===
Large language models process vast sequences of input tokens by alternating between classical multi-layer perceptron layers and self-attention mechanisms. While the approximation capabilities of perceptrons are relatively well understood, those of attention mechanisms remain less explored. In this talk, I will compare the proof techniques and approximation results associated with these two types of layers, emphasizing key open questions that connect large language models with approximation theory in infinite-dimensional spaces representing input token distributions.
Gabriel Peyré (CNRS, DMA, École Normale Supérieure)
===
Find this and many more scientific videos on carmin.tv - a French video platform for mathematics and their interactions with other sciences offering extra functionalities tailored to meet the needs of the research community.
===
