Machine Learning Initiative

In this talk, I will survey recent developments about the connection between neural networks and models of quantum mechanics and quantum field theory. Previous work has shown that the neural network - Gaussian process correspondence can be interpreted as the statement that large-width neural networks share some properties with free, or weakly interacting, quantum field theories (QFTs). Here I will focus on 1d QFTs, or models of quantum mechanics, where one has greater theoretical control. For instance, under mild assumptions, one can prove that any model of a quantum particle admits a representation as a neural network. Cherished features of quantum mechanics, such as uncertainty relations, emerge from specific architectural choices that are made to satisfy the axioms of quantum theory. Based on 2504.05462 with Jim Halverson.

We present a tensorization algorithm for constructing tensor train representations of functions, drawing on sketching and cross interpolation ideas. The method only requires black-box access to the target function and a small set of sample points defining the domain of interest. Thus, it is particularly well-suited for machine learning models, where the domain of interest is naturally defined by the training dataset. We show that this approach can be used to enhance the privacy and interpretability of neural network models. Specifically, we apply our decomposition to (i) obfuscate neural networks whose parameters encode patterns tied to the training data distribution, and (ii) estimate topological phases of matter that are easily accessible from the tensor train representation. Additionally, we show that this tensorization can serve as an efficient initialization method for optimizing tensor trains in general settings, and that, for model compression, our algorithm achieves a superior trade-off between memory and time complexity compared to conventional tensorization methods of neural networks.

In this talk I will give updates on two projects: Firstly, Perimeter and Two Small Fish Ventures are currently supporting a group of five PSI alumni in writing a play themed around quantum science & technology, with the goal of enriching public discourse on these fields. On the one hand, we aim to inform the audience about key quantum concepts in an entertaining setting. On the other hand, we are exploring questions such as "Will we really have fault-tolerant quantum computers in five years? Which quantum research is better pursued in industry vs. academia? Who will benefit from commercialization? What drives scientists to do quantum research - for scientific understanding, or because 'today's theoretical physics is tomorrow's technology'?". Five characters debate different takes on these questions in a Douglas-Adams-inspired sci-fi setting.

Secondly, I will review recent developments in classical and quantum reservoir computing.

Pairwise difference learning (PDL) has recently been introduced as a new meta-learning technique for regression by Wetzel et al. Instead of learning a mapping from instances to outcomes in the standard way, the key idea is to learn a function that takes two instances as input and predicts the difference between the respective outcomes. Given a function of this kind, predictions for a query instance are derived from every training example and then averaged. This presentation focus on the classification version of PDL, proposing a meta-learning technique for inducing a classifier by solving a suitably defined (binary) classification problem on a paired version of the original training data. This presentation will also discuss an enhancement to PDL through anchor weighting, which adjusts the influence of anchor points based on the reliability and precision of their predictions, thus improving the robustness and accuracy of the method.  We analyze the performance of the PDL classifier in a large-scale empirical study, finding that it outperforms state-of-the-art methods in terms of prediction performance. Finally, we provide an easy-to-use and publicly available implementation of PDL in a Python package.

The physics of strongly correlated systems offers some of the most intriguing physics challenges such as competing orders or the emergence of dynamical composite degrees of freedom. Often, the resolution of these physics challenges is computationally hard, but can be simplified by a formulation in terms of the appropriate dynamical degrees of freedom.

We develop a Machine-Learning Renormalization Group (MLRG) algorithm to explore and analyze many-body lattice models in statistical physics. Using the representation learning capability of generative modeling, MLRG automatically learns the optimal renormalization group (RG) transformations from self-generated spin configurations and formulates RG equations without human supervision. The algorithm does not focus on simulating any particular lattice model but broadly explores all possible models compatible with the internal and lattice symmetries given the on-site symmetry representation. It can uncover the RG monotone that governs the RG flow, assuming a strong form of the $c$-theorem. This enables several downstream tasks, including unsupervised classification of phases, automatic location of phase transitions or critical points, controlled estimation of critical exponents, and operator scaling dimensions. We demonstrate the MLRG method in two-dimensional lattice models with Ising symmetry and show that the algorithm correctly identifies and characterizes the Ising criticality.

---

Zoom link

Large neural networks are often studied analytically through scaling limits: regimes in which some structural network parameters (e.g. depth, width, number of training datapoints, and so on) tend to infinity. Such limits are challenging to identify and study in part because the limits as these structural parameters diverge typically do not commute. I will present some recent and ongoing work with Alexander Zlokapa (MIT), in which we provide the first solvable models of learning – in this case by Bayesian inference – with neural networks where the depth, width, and number of datapoints can all be large.

---

Zoom link

Both the social and natural world are replete with complex structure that often has a probabilistic interpretation. In the former, we may seek to model, for example, the distribution of natural images or language, for which there are copious amounts of real world data. In the latter, we are given the probabilistic rule describing a physical process, but no procedure for generating samples under it necessary to perform simulation. In this talk, I will discuss a generative modeling paradigm based on maps between probability distributions that is applicable to both of these circumstances. I will describe a means for learning these maps in the context of problems in statistical physics, how to impose symmetries on them to facilitate learning, and how to use the resultant generative models in a statistically unbiased fashion. I will then describe a paradigm that unifies flow-based and diffusion based generative models by recasting generative modeling as a problem of regression. I will demonstrate the efficacy of doing this in computer vision problems and end with some future challenges and applications.

---

Zoom link

Ultra-cold Fermi gases exhibit a rich array of quantum mechanical properties, including the transition from a fermionic superfluid Bardeen-Cooper-Schrieffer (BCS) state to a bosonic superfluid Bose-Einstein condensate (BEC), which can be precisely probed experimentally. However, accurately describing these properties poses significant theoretical challenges due to strong pairing correlations and non-perturbative interactions. In this talk, I will discuss our recent development—a Pfaffian-Jastrow neural-network quantum state equipped with a message-passing architecture, designed to efficiently capture pairing and backflow correlations. We benchmark our approach against existing Slater-Jastrow frameworks and state-of-the-art diffusion Monte Carlo methods. Analysis of pair distribution functions and pairing gaps reveals the emergence of strong pairing correlations around unitarity. We demonstrate that transfer learning stabilizes the training process in the presence of strong, short-ranged interactions, allowing for an effective exploration of the BCS-BEC crossover region. Our findings highlight the potential of neural-network quantum states as a promising strategy for investigating ultra-cold Fermi gases.

---

Zoom link

Neural Network Field Theories (NNFTs) are field theories defined via output ensembles of initialized Neural Network (NN) architectures, the backbones of current state-of-the-art Deep Learning techniques. Different limits of NN architectures correspond to free, weakly interacting, and non-perturbative regimes of NNFTs, via central limit theorem and its violations. Nature of field interactions in NNFTs can be controlled by tuning architecture parameters and hyperparameters, systematically, at initialization. I will present a systematic construction of scalar NNFT actions, using various attributes of NN architectures, via a new set of Feynman rules and techniques from statistical physics. Conversely, I will present the construction of a class of NN architectures exactly corresponding to some interacting scalar field theories, via a systematic deformation of NN parameter distributions. As an example of the latter method, I will present the construction of an architecture for $\lambda \phi^4$ scalar NNFT. Lastly, I will introduce Grassmann NNFTs, their free and interacting regimes via central limit theorem for Grassmanns, and construction of an architecture corresponding to free Dirac NNFT. This approach provides us a way to initialize NN architectures exactly representing certain field configurations, and are useful for computing attributes, e.g. correlators, of field theories on lattice.

---

Zoom link

I introduce a unified framework for interpreting neural network classifiers tailored toward automated scientific discovery. In contrast to neural network-based regression, for classification, it is in general impossible to find a one-to-one mapping from the neural network to a symbolic equation even if the neural network itself bases its classification on a quantity that can be written as a closed-form equation. In this paper, I embed a trained neural network into an equivalence class of classifying functions that base their decisions on the same quantity. I interpret neural networks by finding an intersection between this equivalence class and human-readable equations defined by the search space of symbolic regression. The approach is not limited to classifiers or full neural networks and can be applied to arbitrary neurons in hidden layers or latent spaces or to simplify the process of interpreting neural network regressors.

---

Zoom link