18848

Video URL

https://simons.berkeley.edu/talks/spectrum-nonlinear-random-matrices-ultra-wide-neural-networks

The Spectrum of Nonlinear Random Matrices for Ultra-Wide Neural Networks

(2021). The Spectrum of Nonlinear Random Matrices for Ultra-Wide Neural Networks. The Simons Institute for the Theory of Computing. https://simons.berkeley.edu/talks/spectrum-nonlinear-random-matrices-ultra-wide-neural-networks

The Spectrum of Nonlinear Random Matrices for Ultra-Wide Neural Networks. The Simons Institute for the Theory of Computing, Dec. 07, 2021, https://simons.berkeley.edu/talks/spectrum-nonlinear-random-matrices-ultra-wide-neural-networks 

          @misc{ scivideos_18848,
            doi = {},
            url = {https://simons.berkeley.edu/talks/spectrum-nonlinear-random-matrices-ultra-wide-neural-networks},
            author = {},
            keywords = {},
            language = {en},
            title = {The Spectrum of Nonlinear Random Matrices for Ultra-Wide Neural Networks},
            publisher = {The Simons Institute for the Theory of Computing},
            year = {2021},
            month = {dec},
            note = {18848 see, \url{https://scivideos.org/index.php/Simons-Institute/18848}}
          }
Yizhe Zhu (University of California, Irvine)
Talk number18848
Source RepositorySimons Institute
Subject

Abstract

We obtain limiting spectral distribution of empirical conjugate kernel and neural tangent kernel matrices for two-layer neural networks with deterministic data and random weights. When the width of the network grows faster than the size of the dataset, a deformed semicircle law appears. In this regime, we also calculate the asymptotic test and training errors for random feature regression. Joint work with Zhichao Wang (UCSD).