PIRSA:26050047

Video URL

https://pirsa.org/26050047

Towards a Science of AI: Scaling laws and synthetic data

Barkeshli, M. (2026). Towards a Science of AI: Scaling laws and synthetic data. Perimeter Institute for Theoretical Physics. https://pirsa.org/26050047

Barkeshli, Maissam. Towards a Science of AI: Scaling laws and synthetic data. Perimeter Institute for Theoretical Physics, May. 05, 2026, https://pirsa.org/26050047 

          @misc{ scivideos_PIRSA:26050047,
            doi = {10.48660/26050047},
            url = {https://pirsa.org/26050047},
            author = {Barkeshli, Maissam},
            keywords = {Other Physics},
            language = {en},
            title = {Towards a Science of AI: Scaling laws and synthetic data},
            publisher = {Perimeter Institute for Theoretical Physics},
            year = {2026},
            month = {may},
            note = {PIRSA:26050047 see, \url{https://scivideos.org/pirsa/26050047}}
          }

Maissam Barkeshli University of California, Santa Barbara

Talk numberPIRSA:26050047
DOI10.48660/26050047
Source RepositoryPIRSA
Collection
Talk Type Scientific Series
Subject

Abstract

The stunning capabilities of modern AI systems give rise to many questions regarding how they work and how much more capable they can possibly get. One way to gain additional insight is via synthetic models of data with tunable complexity, which can capture the basic relevant structures of real data. In recent work we have focused on sequences obtained from random walks on graphs, hypergraphs, and hierarchical graphical structures. I will present some recent empirical results regarding how transformers learn sequences arising from random walks on graphs. The focus will be on neural scaling laws, unexpected temperature-dependent effects, and sample complexity. If there is time, I will also discuss the effect of parameterization strategies on hyperparameter scaling laws, where we see the critical importance of appropriately scaling the embedding layer learning rate.