Uploaded July 2026 | Updated September 2026, 3 weeks ago
Swap compute for data on the scaling curve and the same money buys a better model, which is why Ari Morcos calls data quality the compute multiplier and the most underinvested part of training. His frame is an oil refinery for data rather than a firehose: clean, curate, create, and compose, with quality classifiers, deduplication, and synthetic generation each earning their place, and the sequencing across stages mattering as much as any single step. The scarce resource now is not tokens but signal per token, and finding data that is optimal for a given target is where the leverage hides.
The proof points are concrete. Better curated data lets a small multilingual model beat far larger ones trained on many more tokens, and it buys real inference efficiency because a model reaches the same quality with less. Morcos points to DatologyAI's customer results, from Thomson Reuters gaining on proprietary legal data in mid-training to Arcee's Trinity reaching the open frontier on public data alone. The closing argument is blunt: it is cheaper to manufacture high quality data than to buy more compute, so data curation is quietly shaping the future of model training.
Speaker info:
- https://x.com/arimorcos
- linkedin.com/in/arimorcos
- arimorcos.com
Timestamps:
0:00 - Data is all we think about
0:52 - Why good data became scarce
2:19 - Swapping compute for data on the curve
3:48 - An oil refinery for data
5:52 - Curation work at DatologyAI
6:54 - Proof: small models beating bigger ones
8:58 - Inference efficiency from better data
9:24 - Multilingual gains
12:16 - Synthetic data done right
14:10 - Thomson Reuters and Arcee results
17:43 - Cheaper than buying compute
Swap compute for data on the scaling curve and the same money buys a better model, which is why Ari Morcos calls data quality the compute multiplier and the most underinvested part of training. His frame is an oil refinery for data rather than a firehose: clean, curate, create, and compose, with quality classifiers, deduplication, and synthetic generation each earning their place, and the sequencing across stages mattering as much as any single step. The scarce resource now is not tokens but signal per token, and finding data that is optimal for a given target is where the leverage hides.
The proof points are concrete. Better curated data lets a small multilingual model beat far larger ones trained on many more tokens, and it buys real inference efficiency because a model reaches the same quality with less. Morcos points to DatologyAI's customer results, from Thomson Reuters gaining on proprietary legal data in mid-training to Arcee's Trinity reaching the open frontier on public data alone. The closing argument is blunt: it is cheaper to manufacture high quality data than to buy more compute, so data curation is quietly shaping the future of model training.
Speaker info:
- https://x.com/arimorcos
- linkedin.com/in/arimorcos
- arimorcos.com
Timestamps:
0:00 - Data is all we think about
0:52 - Why good data became scarce
2:19 - Swapping compute for data on the curve
3:48 - An oil refinery for data
5:52 - Curation work at DatologyAI
6:54 - Proof: small models beating bigger ones
8:58 - Inference efficiency from better data
9:24 - Multilingual gains
12:16 - Synthetic data done right
14:10 - Thomson Reuters and Arcee results
17:43 - Cheaper than buying compute
