Uploaded November 2021 | Updated September 2026, 1 day ago
Show notes: braininspired.co/podcast/120
Patreon support: patreon.com/braininspired
Free Video Series: Open Questions in AI and Neuroscience:
braininspired.co/open
James, Andrew, and Weinan discuss their recent theory about how the brain might use complementary learning systems to optimize our memories. The idea is that our hippocampus creates our episodic memories for individual events, full of particular details. And through a complementary process, slowly consolidates those memories within our neocortex through mechanisms like hippocampal replay. The new idea in their work suggests a way for the consolidated cortical memory to become optimized for generalization, something humans are known to be capable of but deep learning has yet to build. We discuss what their theory predicts about how the "correct" process depends on how much noise and variability there is in the learning environment, how their model solves this, and how it relates to our brain and behavior.
0:00 - Intro
3:57 - Guest Intros
15:04 - Organizing memories for generalization
26:48 - Teacher, student, and notebook models
30:51 - Shallow linear networks
33:17 - How to optimize generalization
47:05 - Replay as a generalization regulator
54:57 - Whole greater than sum of its parts
1:05:37 - Unpredictability
1:10:41 - Heuristics
1:13:52 - Theoretical neuroscience for AI
1:29:42 - Current personal thinking
Show notes: braininspired.co/podcast/120
Patreon support: patreon.com/braininspired
Free Video Series: Open Questions in AI and Neuroscience:
braininspired.co/open
James, Andrew, and Weinan discuss their recent theory about how the brain might use complementary learning systems to optimize our memories. The idea is that our hippocampus creates our episodic memories for individual events, full of particular details. And through a complementary process, slowly consolidates those memories within our neocortex through mechanisms like hippocampal replay. The new idea in their work suggests a way for the consolidated cortical memory to become optimized for generalization, something humans are known to be capable of but deep learning has yet to build. We discuss what their theory predicts about how the "correct" process depends on how much noise and variability there is in the learning environment, how their model solves this, and how it relates to our brain and behavior.
0:00 - Intro
3:57 - Guest Intros
15:04 - Organizing memories for generalization
26:48 - Teacher, student, and notebook models
30:51 - Shallow linear networks
33:17 - How to optimize generalization
47:05 - Replay as a generalization regulator
54:57 - Whole greater than sum of its parts
1:05:37 - Unpredictability
1:10:41 - Heuristics
1:13:52 - Theoretical neuroscience for AI
1:29:42 - Current personal thinking
