Uploaded April 2025 | Updated September 2026, 4 days ago
For years it’s been an open question — how much is a language model learning and synthesizing information, and how much is it just memorizing and reciting?
Introducing OLMoTrace, a new feature in the Ai2 Playground that begins to shed some light.
OLMoTrace connects phrases or even whole sentences in the language model’s output back to verbatim matches in its training data. It does this by searching billions of documents and trillions of tokens in real time and highlighting where it finds compelling matches.
You can try OLMoTrace for yourself at playground.allenai.org/!
Blog post: allenai.org/blog/olmotrace
Training and toolkit code: github.com/allenai/infinigram-api
Questions? Ask on Discord: discord.com/invite/NE5xPufNwu
For years it’s been an open question — how much is a language model learning and synthesizing information, and how much is it just memorizing and reciting?
Introducing OLMoTrace, a new feature in the Ai2 Playground that begins to shed some light.
OLMoTrace connects phrases or even whole sentences in the language model’s output back to verbatim matches in its training data. It does this by searching billions of documents and trillions of tokens in real time and highlighting where it finds compelling matches.
You can try OLMoTrace for yourself at playground.allenai.org/!
Blog post: allenai.org/blog/olmotrace
Training and toolkit code: github.com/allenai/infinigram-api
Questions? Ask on Discord: discord.com/invite/NE5xPufNwu
