Uploaded April 2024 | Updated September 2026, 13 hours ago
Please find a copy of the slides at drive.google.com/file/d/1qGHQPkcidivq284OuI5jfL9_KMovcuGA/view?usp=share_link
Despite the widespread proliferation of neural networks, the mechanisms through which they operate so successfully are not well understood. In this talk, we will first explore empirical and theoretical investigations into neural network training and generalization and what they can tell us about why deep learning works. Then, we will examine a recent line of work on algorithm learning. While neural networks typically excel at pattern matching tasks, we consider whether neural networks can learn algorithms that scale to problem instances orders of magnitude larger than those seen during training.
Micah Goldblum is a postdoctoral researcher at New York University with Yann LeCun and Andrew Gordon Wilson. His research portfolio includes award winning work in Bayesian inference, generalization theory, algorithmic reasoning, and AI security and privacy. Micah’s paper on model comparison received the Outstanding Paper Award at ICML 2022. Before his current position, he received a Ph.D. in mathematics at the University of Maryland where he worked with Tom Goldstein and Wojciech Czaja.
0:00-2:20 Intro
2:20-3:45 Micah Goldblum and Deep Learning
3:45 - 18:40 Generalization and Extrema
18:40 - 21:51 Why does Generalization happen?
21:51 - 26:42 Generalization Bounds
26:42 - 30:18 How to craft tight bounds
30:18 - 32:25 Theory
32:25 - 39:40 Marginal Likelihood & Generalization
39:40 - 40:38 Concept Wrap-Up
40:38 - 42:05 How far does it go? Why?
42:05 - 49:47 Recurrent & Feed Forward Training
49:47 - 51:45 Some thoughts
51:45 - 1:00:47 Q&A
1:00:47 - 1:01:26 Summary and wrap up
Please find a copy of the slides at drive.google.com/file/d/1qGHQPkcidivq284OuI5jfL9_KMovcuGA/view?usp=share_link
Despite the widespread proliferation of neural networks, the mechanisms through which they operate so successfully are not well understood. In this talk, we will first explore empirical and theoretical investigations into neural network training and generalization and what they can tell us about why deep learning works. Then, we will examine a recent line of work on algorithm learning. While neural networks typically excel at pattern matching tasks, we consider whether neural networks can learn algorithms that scale to problem instances orders of magnitude larger than those seen during training.
Micah Goldblum is a postdoctoral researcher at New York University with Yann LeCun and Andrew Gordon Wilson. His research portfolio includes award winning work in Bayesian inference, generalization theory, algorithmic reasoning, and AI security and privacy. Micah’s paper on model comparison received the Outstanding Paper Award at ICML 2022. Before his current position, he received a Ph.D. in mathematics at the University of Maryland where he worked with Tom Goldstein and Wojciech Czaja.
0:00-2:20 Intro
2:20-3:45 Micah Goldblum and Deep Learning
3:45 - 18:40 Generalization and Extrema
18:40 - 21:51 Why does Generalization happen?
21:51 - 26:42 Generalization Bounds
26:42 - 30:18 How to craft tight bounds
30:18 - 32:25 Theory
32:25 - 39:40 Marginal Likelihood & Generalization
39:40 - 40:38 Concept Wrap-Up
40:38 - 42:05 How far does it go? Why?
42:05 - 49:47 Recurrent & Feed Forward Training
49:47 - 51:45 Some thoughts
51:45 - 1:00:47 Q&A
1:00:47 - 1:01:26 Summary and wrap up
