Uploaded April 2022 | Updated September 2026, 9 hours ago
Chinchilla is a massive language released by DeepMind as part of a recent paper that focuses on scaling large language models in a compute-optimal manner. It outperforms recent models like GPT-3, Gopher, and Megatron-Turing NLG that use hundreds of billions of parameters with only 70 billion parameters. They achieve this by training 400 large models to find the optimal ratio of parameters and amount of training data to train a model given a computation budget.
Outline:
0:00 - Overview
1:51 - Paper Intro
6:15 - Methods
18:14 - Scaling Implications
23:43 - Chinchilla Overview
25:48 - Chinchilla Performance
29:49 - Summary
30:07 - Thoughts & Critiques
Paper (Training Compute-Optimal Large Language Models): arxiv.org/abs/2203.15556
Chinchilla is a massive language released by DeepMind as part of a recent paper that focuses on scaling large language models in a compute-optimal manner. It outperforms recent models like GPT-3, Gopher, and Megatron-Turing NLG that use hundreds of billions of parameters with only 70 billion parameters. They achieve this by training 400 large models to find the optimal ratio of parameters and amount of training data to train a model given a computation budget.
Outline:
0:00 - Overview
1:51 - Paper Intro
6:15 - Methods
18:14 - Scaling Implications
23:43 - Chinchilla Overview
25:48 - Chinchilla Performance
29:49 - Summary
30:07 - Thoughts & Critiques
Paper (Training Compute-Optimal Large Language Models): arxiv.org/abs/2203.15556
![Automating Research With GPT API [Livestream]](https://i.ytimg.com/vi/VnZpmDaWpEc/mqdefault.jpg "Automating Research With GPT API [Livestream]
The title says it all, Im figuring this out as I go! Currently Im using GPT-3.5 turbo to do all this testing.
The current plan is to create a system that:
1. Can generate research ideas in a target area
2. Use the idea to make a list of proof of concept experiments
3. Write the code for the experiments
4. Debug")
