Zico Kolter: Deep neural networks via monotone operators @InternationalMathematicalUnion
Zico Kolter: Deep neural networks via monotone operators  @InternationalMathematicalUnion
Uploaded July 2022 | Updated November 2026, 3 days ago
Deep Equilibrium Models (DEQs) are a class of machine learning models that computes the fixed point of a single nonlinear operator in lieu of a traditional multi-layer network. Our recently proposed monotone equilibrium model (monDEQ) provides a firmer grounding for such models, by providing a parameterization that guarantees the existence and uniqueness of this fixed point. In this talk, I will highlight some of our recent work on these monDEQ models. Specifically, I will first highlight the general monDEQ framework, how it provides guarantees on fixed point uniqueness, and how it allows for efficient operator splitting methods for computing the fixed point; I will then illustrate how it enables us bound Lipchitz constant of the monDEQ models, which in turn allows us to produce generalization bounds for these architectures (non-trivial for these effectively "infinitely deep" networks); I will highlight a few different approaches to characterizing robustness of monDEQ models; and I will finish by highlighting the connections between monDEQs and mean-field inference, to illustrate how we can use these approaches to formulate Boltzman-machine-like models for which mean-field inference is guaranteed to converge to a globally optimal solution.

Slides: mathunion.org/fileadmin/IMU/ICM2022/Presentation-slides/183-Zico%20Kolter.pdf
Zico Kolter: Deep neural networks via monotone operatorsMiklós Abert: Groups, manifolds and graph limitsPatricia Gonçalves: On hydrodynamic limits of fractional PDEs from stochastic interacting...Binyong Sun, Chen-Bo Zhu: Theta correspondence and the orbit methodChi Li: Canonical Kaehler metrics and stability of algebraic varietiesDimitris Koukoulopoulos: Rational approximations of irrational numbersJian Ding, Julien Dubedat, Ewain Gwynne: The Liouville quantum gravity metricBenoit Collins: Weingarten calculus and its applicationsZhifei Zhang: Hydrodynamic stability at high Reynolds number and Transition threshold problemRomain Dujardin: Random algebraic dynamics in complex dimension 2Carlos E. Kenig - Closing RemarksDmitry Novikov, Gal Binyamini: Tameness in geometry and arithmetic: beyond o-minimality
International Mathematical Union |

Zico Kolter: Deep neural networks via monotone operators

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER