Reinventing Entropy | Compression is Intelligence Part 1 @3blue1brown
Reinventing Entropy | Compression is Intelligence Part 1  @3blue1brown
Uploaded June 2026 | Updated November 2026, 2 days ago
How to study the compressibility of language.
Check out our virtual career fair: 3b1b.co/talent
See new projects before they go live: 3b1b.co/support

Animation credit:
Manim scenes by Aaron Gostein and Grant Sanderson
Shannon’s story, as well as those for various pi creatures, by Mitchell Zemil.
Lunar robot and prediction/compression coin by Paul Dancstep
NanoGPT animations by Clayton Rabideau

The way of visualizing entropy shown here is something I first came across in this excellent post by Chris Olah:
colah.github.io/posts/2015-09-Visual-Information

Shannon’s “A Mathematical Theory of Communication”
https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf

Shannon’s “Prediction and Entropy of Printed English”
https://www.princeton.edu/~wbialek/rome/refs/shannon_51.pdf

Scientific American article that mentions the story with Von Neumann suggesting the name Entropy:
https://www.esalq.usp.br/lepse/imgs/conteudo_thumb/Energy-and-Information.pdf


Timestamps:

0:00 - On “Compression is intelligence.”
3:28 - The warmup example
10:46 - What perfect compression looks like
14:47 - Defining information
17:40 - Information of language
24:29 - Defining Entropy
31:14 - 3b1b Talent

------------------

These animations are largely made using a custom Python library, manim. See the FAQ comments here:
3b1b.co/faq#manim

Music by Vincent Rubinetti.
vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u

------------------

3blue1brown is a channel about animating math, in all senses of the word animate. If you're reading the bottom of a video description, I'm guessing you're more interested than the average viewer in lessons here. It would mean a lot to me if you chose to stay up to date on new ones, either by subscribing here on YouTube or otherwise following on whichever platform below you check most regularly.

Mailing list: 3blue1brown.substack.com
Twitter: twitter.com/3blue1brown
Bluesky: https://bsky.app/profile/3blue1brown.com
Instagram: instagram.com/3blue1brown
Reddit: reddit.com/r/3blue1brown
Facebook: facebook.com/3blue1brown
Patreon: patreon.com/3blue1brown
Website: 3blue1brown.com
Reinventing Entropy | Compression is Intelligence Part 1The medical test paradox, and redesigning Bayes ruleHow (and why) to take a logarithm of an imageQ&A #2 + Net Neutrality NuanceA tale of two problem solvers | Average cube shadow areaBut what is a partial differential equation?  | DE2Group theory, abstraction, and the 196,883-dimensional monsterThe tragic tale of Guillaume Le GentilThis puzzle is trickier than it seemsFourier seriesMake math videos! | Summer of Math Exposition announcementInfinite Lighthouses and π
3Blue1Brown |

Reinventing Entropy | Compression is Intelligence Part 1

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER