No Language Left Behind  Unlocking Text Data for Under Resourced | AI2 @allenai
No Language Left Behind  Unlocking Text Data for Under Resourced | AI2  @allenai
Uploaded February 2023 | Updated November 2026, 3 days ago
No Language Left Behind Unlocking Text Data for Under Resourced
Shruti Rijhwani

NLP systems are limited by the availability of text data, and because machine-readable text exists only in a few hundred languages, most of the world’s languages are under-represented in modern language technologies.
Text data exists in many more languages! However, it is locked away in printed books and handwritten documents, and training a high-performance optical character recognition (OCR) system to extract the text is challenging for most under-resourced languages.

In this talk, I will describe two methods for improving text recognition in low-resource settings using automatic OCR post-correction. The first is a multi-source encoder-decoder model with structural biases to efficiently learn from limited data. The second is a semi-supervised learning technique that uses raw unlabeled images to improve performance without additional manual annotation. The method combines self-training with automatically derived lexica through the use of weighted finite-state automata (WFSA) to improve post-correction. I will present empirical evaluation on multiple under-resourced languages to illustrate the effectiveness of the proposed approaches as well as future applications in using the methods to extract text from historical documents in a variety of domains and languages.
No Language Left Behind  Unlocking Text Data for Under Resourced | AI2Towards General Purpose Vision SystemsOptimization within Latent SpacesToyBlocks: Our first Foundation World for robotics learning and evaluationLearning Language-Guided Visuomotor Policies for Robotic ManipulationAi2 OLMoE iOS app: Fully open source, running entirely on-deviceRole of Large Language Models in Human-AI Interaction: A Critical AppraisalMachine Learning in Climate ActionMolmoWeb in Action👋 Meet Molmo: A Family of Open State-of-the-Art Multimodal AI ModelsHey AI, Can You Solve Complex Tasks by Talking to Agents?Time Waits for No One! Analysis and Challenges of Temporal Misalignment
Ai2 |

No Language Left Behind Unlocking Text Data for Under Resourced | AI2

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER