Geometry and descent in random landscapes

Abstract

From quenched physical systems to simple optimization algorithms, descent in an energy or cost landscape is thought to underpin many interesting phenomena. Likewise, the geometry of this landscape is thought to explain diverse outcomes of this descent: in those with few minima, the lowest are easily found; while in those with very many, getting stuck among a proliferation of high minima is overwhelmingly likely. In mean-field spin glasses and inference problems, we can hope to use geometry to explain when transitions between these behaviors occur and which kind of minima will typically be found. I will describe the past success of this approach in the simplest spin glass models and its recent failure in slightly less simple ones. I will share several new results on the statistics and arrangement of metastable states and saddle points which together indicate that traditional mean-field geometric tools may not be able to answer everything. Finally, I share some thoughts on what extensions might be needed to precisely predict where descent goes.