Machine Learning Initiative

In this talk I will give updates on two projects: Firstly, Perimeter and Two Small Fish Ventures are currently supporting a group of five PSI alumni in writing a play themed around quantum science & technology, with the goal of enriching public discourse on these fields. On the one hand, we aim to inform the audience about key quantum concepts in an entertaining setting. On the other hand, we are exploring questions such as "Will we really have fault-tolerant quantum computers in five years? Which quantum research is better pursued in industry vs. academia? Who will benefit from commercialization? What drives scientists to do quantum research - for scientific understanding, or because 'today's theoretical physics is tomorrow's technology'?". Five characters debate different takes on these questions in a Douglas-Adams-inspired sci-fi setting.

Secondly, I will review recent developments in classical and quantum reservoir computing.

Pairwise difference learning (PDL) has recently been introduced as a new meta-learning technique for regression by Wetzel et al. Instead of learning a mapping from instances to outcomes in the standard way, the key idea is to learn a function that takes two instances as input and predicts the difference between the respective outcomes. Given a function of this kind, predictions for a query instance are derived from every training example and then averaged. This presentation focus on the classification version of PDL, proposing a meta-learning technique for inducing a classifier by solving a suitably defined (binary) classification problem on a paired version of the original training data. This presentation will also discuss an enhancement to PDL through anchor weighting, which adjusts the influence of anchor points based on the reliability and precision of their predictions, thus improving the robustness and accuracy of the method.  We analyze the performance of the PDL classifier in a large-scale empirical study, finding that it outperforms state-of-the-art methods in terms of prediction performance. Finally, we provide an easy-to-use and publicly available implementation of PDL in a Python package.

The physics of strongly correlated systems offers some of the most intriguing physics challenges such as competing orders or the emergence of dynamical composite degrees of freedom. Often, the resolution of these physics challenges is computationally hard, but can be simplified by a formulation in terms of the appropriate dynamical degrees of freedom.

We develop a Machine-Learning Renormalization Group (MLRG) algorithm to explore and analyze many-body lattice models in statistical physics. Using the representation learning capability of generative modeling, MLRG automatically learns the optimal renormalization group (RG) transformations from self-generated spin configurations and formulates RG equations without human supervision. The algorithm does not focus on simulating any particular lattice model but broadly explores all possible models compatible with the internal and lattice symmetries given the on-site symmetry representation. It can uncover the RG monotone that governs the RG flow, assuming a strong form of the $c$-theorem. This enables several downstream tasks, including unsupervised classification of phases, automatic location of phase transitions or critical points, controlled estimation of critical exponents, and operator scaling dimensions. We demonstrate the MLRG method in two-dimensional lattice models with Ising symmetry and show that the algorithm correctly identifies and characterizes the Ising criticality.

---

Zoom link

Large neural networks are often studied analytically through scaling limits: regimes in which some structural network parameters (e.g. depth, width, number of training datapoints, and so on) tend to infinity. Such limits are challenging to identify and study in part because the limits as these structural parameters diverge typically do not commute. I will present some recent and ongoing work with Alexander Zlokapa (MIT), in which we provide the first solvable models of learning – in this case by Bayesian inference – with neural networks where the depth, width, and number of datapoints can all be large.

---

Zoom link

Both the social and natural world are replete with complex structure that often has a probabilistic interpretation. In the former, we may seek to model, for example, the distribution of natural images or language, for which there are copious amounts of real world data. In the latter, we are given the probabilistic rule describing a physical process, but no procedure for generating samples under it necessary to perform simulation. In this talk, I will discuss a generative modeling paradigm based on maps between probability distributions that is applicable to both of these circumstances. I will describe a means for learning these maps in the context of problems in statistical physics, how to impose symmetries on them to facilitate learning, and how to use the resultant generative models in a statistically unbiased fashion. I will then describe a paradigm that unifies flow-based and diffusion based generative models by recasting generative modeling as a problem of regression. I will demonstrate the efficacy of doing this in computer vision problems and end with some future challenges and applications.

---

Zoom link

Ultra-cold Fermi gases exhibit a rich array of quantum mechanical properties, including the transition from a fermionic superfluid Bardeen-Cooper-Schrieffer (BCS) state to a bosonic superfluid Bose-Einstein condensate (BEC), which can be precisely probed experimentally. However, accurately describing these properties poses significant theoretical challenges due to strong pairing correlations and non-perturbative interactions. In this talk, I will discuss our recent development—a Pfaffian-Jastrow neural-network quantum state equipped with a message-passing architecture, designed to efficiently capture pairing and backflow correlations. We benchmark our approach against existing Slater-Jastrow frameworks and state-of-the-art diffusion Monte Carlo methods. Analysis of pair distribution functions and pairing gaps reveals the emergence of strong pairing correlations around unitarity. We demonstrate that transfer learning stabilizes the training process in the presence of strong, short-ranged interactions, allowing for an effective exploration of the BCS-BEC crossover region. Our findings highlight the potential of neural-network quantum states as a promising strategy for investigating ultra-cold Fermi gases.

---

Zoom link

Neural Network Field Theories (NNFTs) are field theories defined via output ensembles of initialized Neural Network (NN) architectures, the backbones of current state-of-the-art Deep Learning techniques. Different limits of NN architectures correspond to free, weakly interacting, and non-perturbative regimes of NNFTs, via central limit theorem and its violations. Nature of field interactions in NNFTs can be controlled by tuning architecture parameters and hyperparameters, systematically, at initialization. I will present a systematic construction of scalar NNFT actions, using various attributes of NN architectures, via a new set of Feynman rules and techniques from statistical physics. Conversely, I will present the construction of a class of NN architectures exactly corresponding to some interacting scalar field theories, via a systematic deformation of NN parameter distributions. As an example of the latter method, I will present the construction of an architecture for $\lambda \phi^4$ scalar NNFT. Lastly, I will introduce Grassmann NNFTs, their free and interacting regimes via central limit theorem for Grassmanns, and construction of an architecture corresponding to free Dirac NNFT. This approach provides us a way to initialize NN architectures exactly representing certain field configurations, and are useful for computing attributes, e.g. correlators, of field theories on lattice.

---

Zoom link

I introduce a unified framework for interpreting neural network classifiers tailored toward automated scientific discovery. In contrast to neural network-based regression, for classification, it is in general impossible to find a one-to-one mapping from the neural network to a symbolic equation even if the neural network itself bases its classification on a quantity that can be written as a closed-form equation. In this paper, I embed a trained neural network into an equivalence class of classifying functions that base their decisions on the same quantity. I interpret neural networks by finding an intersection between this equivalence class and human-readable equations defined by the search space of symbolic regression. The approach is not limited to classifiers or full neural networks and can be applied to arbitrary neurons in hidden layers or latent spaces or to simplify the process of interpreting neural network regressors.

---

Zoom link 

Isometric tensor networks (isoTNS) form a subclass of tensor network states that have an additional isometric condition, which implies that they can be efficiently prepared with a linear-depth quantum circuit. In this work, we introduce a procedure to construct isoTNS encoding of certain 2D classical partition functions. By continuously tuning a parameter in the isoTNS, the many-body ground state undergoes quantum phase transitions, exhibiting distinct 2D topological order. We illustrate this by constructing an isoTNS path with bond dimension $D = 2$ interpolating between distinct symmetry-enriched topological (SET) phases. At the transition point, the isoTNS wavefunction is related to a gapless point in the classical six-vertex model. Furthermore, the critical wavefunction supports a power-law correlation along one spatial direction while remains long-range ordered in the other spatial direction. We provide an exact linear-depth parametrized local quantum circuit that realizes the path. The above features can therefore be efficiently realized on a programmable quantum device. In the second part of my talk, I will show how to discover efficiently measurable order parameters for quantum phases using model-independent training of quantum circuit classifiers. The possibility of the efficient realization of phase transition path is useful for benchmarking quantum phase recognition methods in higher than one dimension.

---

Zoom link https://pitp.zoom.us/j/93183360141?pwd=RVdYeUxUbE1aZ1dUbzRSL3lBb0lHZz09

Despite their simple intuition, convolutions are more tedious to analyze than dense layers, which complicates the transfer of theoretical and algorithmic ideas. We provide a simplifying perspective onto convolutions through tensor networks (TNs) which allow reasoning about the underlying tensor multiplications by drawing diagrams, and manipulating them to perform function transformations and sub-tensor access. We demonstrate this expressive power by deriving the diagrams of various autodiff operations and popular approximations of second-order information with full hyper-parameter support, batching, channel groups, and generalization to arbitrary convolution dimensions. Further, we provide convolution-specific transformations based on the connectivity pattern which allow to re-wire and simplify diagrams before evaluation. Finally, we probe computational performance, relying on established machinery for efficient TN contraction. Our TN implementation speeds up a recently-proposed KFAC variant up to 4.5x and enables new hardware-efficient tensor dropout for approximate backpropagation.

---

Zoom link https://pitp.zoom.us/j/99090845943?pwd=NHBNVTNnbDNSOGNSVzNGS21xcllFdz09