Mean-field approximations for high-dimensional Bayesian Regression

          @misc{ scivideos_22591,
            doi = {},
            url = {https://old.simons.berkeley.edu/node/22591},
            author = {},
            keywords = {},
            language = {en},
            title = {Mean-field approximations for high-dimensional Bayesian Regression},
            publisher = {The Simons Institute for the Theory of Computing},
            year = {2022},
            month = {sep},
            note = {22591 see, \url{https://scivideos.org/index.php/simons-institute/22591}}
          }
          

Abstract

Abstract Variational approximations provide an attractive computational alternative to MCMC-based strategies for approximating the posterior distribution in Bayesian inference. Despite their popularity in applications, supporting theoretical guarantees are limited, particularly in high-dimensional settings. In this talk, we will study bayesian inference in the context of a linear model with product priors, and derive sufficient conditions for the correctness (to leading order) of the naive mean-field approximation. To this end, we will utilize recent advances in the theory of non-linear large deviations (Chatterjee and Dembo 2014). Next, we analyze the naive mean-field variational problem, and precisely characterize the asymptotic properties of the posterior distribution in this setting. The theory of graph limits provides a crucial ingredient to study this high-dimensional variational problem. This is based on joint work with Sumit Mukherjee (Columbia University).