Unsupervised Brain Models - How does Deep Learning inform Neuroscience? (w/ Patrick Mineault) @YannicKilcher

Uploaded February 2022 | Updated September 2026, 2 weeks ago

#deeplearning #brain #neuroscience

Originally, Deep Learning sprang into existence inspired by how the brain processes information, but the two fields have diverged ever since. However, given that deep models can solve many perception tasks with remarkable accuracy, is it possible that we might be able to learn something about how the brain works by inspecting our models? I speak to Patrick Mineault about his blog post "2021 in review: unsupervised brain models" and we explore why neuroscientists are taking interest in unsupervised and self-supervised deep neural networks in order to explain how the brain works. We discuss a series of influential papers that have appeared last year, and we go into the more general questions of connecting neuroscience and machine learning.

OUTLINE:
0:00 - Intro & Overview
6:35 - Start of Interview
10:30 - Visual processing in the brain
12:50 - How does deep learning inform neuroscience?
21:15 - Unsupervised training explains the ventral stream
30:50 - Predicting own motion parameters explains the dorsal stream
42:20 - Why are there two different visual streams?
49:45 - Concept cells and representation learning
56:20 - Challenging the manifold theory
1:08:30 - What are current questions in the field?
1:13:40 - Should the brain inform deep learning?
1:18:50 - Neuromatch Academy and other endeavours

Blog Post: xcorr.net/2021/12/31/2021-in-review-unsupervised-brain-models
Patrick's Blog: xcorr.net
Twitter: twitter.com/patrickmineault
Neuromatch Academy: academy.neuromatch.io

Links:
TabNine Code Completion (Referral): bit.ly/tabnine-yannick
YouTube: youtube.com/c/yannickilcher
Twitter: twitter.com/ykilcher
Discord: discord.gg/4H8xxDF
BitChute: bitchute.com/channel/yannic-kilcher
LinkedIn: linkedin.com/in/ykilcher
BiliBili: space.bilibili.com/2017636191

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: subscribestar.com/yannickilcher
Patreon: patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n

Unsupervised Brain Models - How does Deep Learning inform Neuroscience? (w/ Patrick Mineault)

$Dynamic Inference with Neural Interpreters (w/ author interview) #deeplearning #neuralinterpreter #ai This video includes an interview with the papers authors! What if we treated deep networks like modular programs? Neural Interpreters divide computation into small modules and route data to them via a dynamic type inference system. The resulting model combines recurrent elements, weight sharing, attention, and more to tackle both abstract reasoning, as well as computer vision tasks. OUTLINE: 0:00 - Intro & Overview 3:00 - Model Overview 7:00 - Interpreter weights and function code 9:40 - Routing data to functions via neural type inference 14:55 - ModLin layers 18:25 - Experiments 21:35 - Interview Start 24:50 - General Model Structure 30:10 - Function code and signature 40:30 - Explaining Modulated Layers 49:50 - A closer look at weight sharing 58:30 - Experimental Results Paper: https://arxiv.org/abs/2110.06399 Guests: Nasim Rahaman: https://twitter.com/nasim_rahaman Francesco Locatello: https://twitter.com/FrancescoLocat8 Waleed Gondal: https://twitter.com/Wallii_gondal Abstract: Modern neural network architectures can leverage large amounts of data to generalize well within the training distribution. However, they are less capable of systematic generalization to data drawn from unseen but related distributions, a feat that is hypothesized to require compositional reasoning and reuse of knowledge. In this work, we present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules, which we call emph{functions}. Inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. The proposed architecture can flexibly compose computation along width and depth, and lends itself well to capacity extension after training. To demonstrate the versatility of Neural Interpreters, we evaluate it in two distinct settings: image classification and visual abstract reasoning on Raven Progressive Matrices. In the former, we show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner. In the latter, we find that Neural Interpreters are competitive with respect to the state-of-the-art in terms of systematic generalization Authors: Nasim Rahaman, Muhammad Waleed Gondal, Shruti Joshi, Peter Gehler, Yoshua Bengio, Francesco Locatello, Bernhard Schölkopf Links: TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick YouTube: https://www.youtube.com/c/yannickilcher Twitter: https://twitter.com/ykilcher Discord: https://discord.gg/4H8xxDF BitChute: https://www.bitchute.com/channel/yannic-kilcher LinkedIn: https://www.linkedin.com/in/ykilcher BiliBili: https://space.bilibili.com/2017636191 If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this): SubscribeStar: https://www.subscribestar.com/yannickilcher Patreon: https://www.patreon.com/yannickilcher Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2 Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n$

RWKV: Reinventing RNNs for the Transformer Era (Paper Explained)

[ML News] Stable Diffusion Takes Over! (Open Source AI Art)

[ML News] This AI completes Wikipedia! Meta AI Sphere | Google Minerva | GPT-3 writes a paper

[ML News] LLaMA2 Released | LLMs for Robots | Multimodality on the Rise

This ChatGPT Skill will earn you $10B (also, AI reads your mind!) | ML News

[ML News] ConvNeXt: Convolutions return | China regulates algorithms | Saliency cropping examined

Author Interview - Improving Intrinsic Exploration with Language Abstractions

[Paper Analysis] On the Theoretical Limitations of Embedding-Based Retrieval (Warning: Rant)

AI against Censorship: Genetic Algorithms, The Geneva Project, ML in Security, and more!