Machine Learning Initiative

We develop a Machine-Learning Renormalization Group (MLRG) algorithm to explore and analyze many-body lattice models in statistical physics. Using the representation learning capability of generative modeling, MLRG automatically learns the optimal renormalization group (RG) transformations from self-generated spin configurations and formulates RG equations without human supervision. The algorithm does not focus on simulating any particular lattice model but broadly explores all possible models compatible with the internal and lattice symmetries given the on-site symmetry representation. It can uncover the RG monotone that governs the RG flow, assuming a strong form of the $c$-theorem. This enables several downstream tasks, including unsupervised classification of phases, automatic location of phase transitions or critical points, controlled estimation of critical exponents, and operator scaling dimensions. We demonstrate the MLRG method in two-dimensional lattice models with Ising symmetry and show that the algorithm correctly identifies and characterizes the Ising criticality.

---

Zoom link

Large neural networks are often studied analytically through scaling limits: regimes in which some structural network parameters (e.g. depth, width, number of training datapoints, and so on) tend to infinity. Such limits are challenging to identify and study in part because the limits as these structural parameters diverge typically do not commute. I will present some recent and ongoing work with Alexander Zlokapa (MIT), in which we provide the first solvable models of learning – in this case by Bayesian inference – with neural networks where the depth, width, and number of datapoints can all be large.

---

Zoom link

Both the social and natural world are replete with complex structure that often has a probabilistic interpretation. In the former, we may seek to model, for example, the distribution of natural images or language, for which there are copious amounts of real world data. In the latter, we are given the probabilistic rule describing a physical process, but no procedure for generating samples under it necessary to perform simulation. In this talk, I will discuss a generative modeling paradigm based on maps between probability distributions that is applicable to both of these circumstances. I will describe a means for learning these maps in the context of problems in statistical physics, how to impose symmetries on them to facilitate learning, and how to use the resultant generative models in a statistically unbiased fashion. I will then describe a paradigm that unifies flow-based and diffusion based generative models by recasting generative modeling as a problem of regression. I will demonstrate the efficacy of doing this in computer vision problems and end with some future challenges and applications.

---

Zoom link

Ultra-cold Fermi gases exhibit a rich array of quantum mechanical properties, including the transition from a fermionic superfluid Bardeen-Cooper-Schrieffer (BCS) state to a bosonic superfluid Bose-Einstein condensate (BEC), which can be precisely probed experimentally. However, accurately describing these properties poses significant theoretical challenges due to strong pairing correlations and non-perturbative interactions. In this talk, I will discuss our recent development—a Pfaffian-Jastrow neural-network quantum state equipped with a message-passing architecture, designed to efficiently capture pairing and backflow correlations. We benchmark our approach against existing Slater-Jastrow frameworks and state-of-the-art diffusion Monte Carlo methods. Analysis of pair distribution functions and pairing gaps reveals the emergence of strong pairing correlations around unitarity. We demonstrate that transfer learning stabilizes the training process in the presence of strong, short-ranged interactions, allowing for an effective exploration of the BCS-BEC crossover region. Our findings highlight the potential of neural-network quantum states as a promising strategy for investigating ultra-cold Fermi gases.

---

Zoom link

Neural Network Field Theories (NNFTs) are field theories defined via output ensembles of initialized Neural Network (NN) architectures, the backbones of current state-of-the-art Deep Learning techniques. Different limits of NN architectures correspond to free, weakly interacting, and non-perturbative regimes of NNFTs, via central limit theorem and its violations. Nature of field interactions in NNFTs can be controlled by tuning architecture parameters and hyperparameters, systematically, at initialization. I will present a systematic construction of scalar NNFT actions, using various attributes of NN architectures, via a new set of Feynman rules and techniques from statistical physics. Conversely, I will present the construction of a class of NN architectures exactly corresponding to some interacting scalar field theories, via a systematic deformation of NN parameter distributions. As an example of the latter method, I will present the construction of an architecture for $\lambda \phi^4$ scalar NNFT. Lastly, I will introduce Grassmann NNFTs, their free and interacting regimes via central limit theorem for Grassmanns, and construction of an architecture corresponding to free Dirac NNFT. This approach provides us a way to initialize NN architectures exactly representing certain field configurations, and are useful for computing attributes, e.g. correlators, of field theories on lattice.

---

Zoom link

I introduce a unified framework for interpreting neural network classifiers tailored toward automated scientific discovery. In contrast to neural network-based regression, for classification, it is in general impossible to find a one-to-one mapping from the neural network to a symbolic equation even if the neural network itself bases its classification on a quantity that can be written as a closed-form equation. In this paper, I embed a trained neural network into an equivalence class of classifying functions that base their decisions on the same quantity. I interpret neural networks by finding an intersection between this equivalence class and human-readable equations defined by the search space of symbolic regression. The approach is not limited to classifiers or full neural networks and can be applied to arbitrary neurons in hidden layers or latent spaces or to simplify the process of interpreting neural network regressors.

---

Zoom link 

Isometric tensor networks (isoTNS) form a subclass of tensor network states that have an additional isometric condition, which implies that they can be efficiently prepared with a linear-depth quantum circuit. In this work, we introduce a procedure to construct isoTNS encoding of certain 2D classical partition functions. By continuously tuning a parameter in the isoTNS, the many-body ground state undergoes quantum phase transitions, exhibiting distinct 2D topological order. We illustrate this by constructing an isoTNS path with bond dimension $D = 2$ interpolating between distinct symmetry-enriched topological (SET) phases. At the transition point, the isoTNS wavefunction is related to a gapless point in the classical six-vertex model. Furthermore, the critical wavefunction supports a power-law correlation along one spatial direction while remains long-range ordered in the other spatial direction. We provide an exact linear-depth parametrized local quantum circuit that realizes the path. The above features can therefore be efficiently realized on a programmable quantum device. In the second part of my talk, I will show how to discover efficiently measurable order parameters for quantum phases using model-independent training of quantum circuit classifiers. The possibility of the efficient realization of phase transition path is useful for benchmarking quantum phase recognition methods in higher than one dimension.

---

Zoom link https://pitp.zoom.us/j/93183360141?pwd=RVdYeUxUbE1aZ1dUbzRSL3lBb0lHZz09

Despite their simple intuition, convolutions are more tedious to analyze than dense layers, which complicates the transfer of theoretical and algorithmic ideas. We provide a simplifying perspective onto convolutions through tensor networks (TNs) which allow reasoning about the underlying tensor multiplications by drawing diagrams, and manipulating them to perform function transformations and sub-tensor access. We demonstrate this expressive power by deriving the diagrams of various autodiff operations and popular approximations of second-order information with full hyper-parameter support, batching, channel groups, and generalization to arbitrary convolution dimensions. Further, we provide convolution-specific transformations based on the connectivity pattern which allow to re-wire and simplify diagrams before evaluation. Finally, we probe computational performance, relying on established machinery for efficient TN contraction. Our TN implementation speeds up a recently-proposed KFAC variant up to 4.5x and enables new hardware-efficient tensor dropout for approximate backpropagation.

---

Zoom link https://pitp.zoom.us/j/99090845943?pwd=NHBNVTNnbDNSOGNSVzNGS21xcllFdz09

In quantum metrology, one of the major applications of quantum technologies, the ultimate precision of estimating an unknown parameter is often stated in terms of the Cramér-Rao bound. Yet, the latter is no longer guaranteed to carry an operational meaning in the regime where few measurement samples are obtained. We instead propose to quantify the quality of a metrology protocol by the probability of obtaining an estimate with a given accuracy. This approach, which we refer to as probably approximately correct (PAC) metrology, ensures operational significance in the finite-sample regime. The accuracy guarantees hold for any value of the unknown parameter, unlike the Cramér-Rao bound which assumes it is approximately known. We establish a strong connection to multi-hypothesis testing with quantum states, which allows us to derive an analogue of the Cramér-Rao bound which contains explicit corrections relevant to the finite-sample regime. We further study the asymptotic behavior of the success probability of the estimation procedure for many copies of the state and apply our framework to the example task of phase estimation with an ensemble of spin-1/2 particles. Overall, our operational approach allows the study of quantum metrology in the finite-sample regime and opens up a plethora of new avenues for research at the interface of quantum information theory and quantum metrology. TL;DR: In this talk, I will motivate why the Cramér-Rao bound might not always be the tool of choice to quantify the ultimate precision attainable in a quantum metrology task and give a (hopefully) intuitive introduction of how we propose to instead quantify it in a way that is valid in the single- and few-shot settings. We will together unearth a strong connection to quantum multi-hypothesis testing and conclude that there are many exiting and fundamental open questions in single-shot metrology!

---

Zoom link https://pitp.zoom.us/j/92247273192?pwd=ZkprOFZ0eEdQYjJDY1hneFNLckFDZz09

Quantum state tomography (QST) is the art of reconstructing an unknown quantum state through measurements. It is a key primitive for developing quantum technologies. Neural network quantum state tomography (NNQST), which aims to reconstruct the quantum state via a neural network ansatz, is often implemented via a basis-dependent cross-entropy loss function. State-of-the-art implementations of NNQST are often restricted to characterizing a particular subclass of states, to avoid an exponential growth in the number of required measurement settings. In this talk, I will discuss an alternative neural-network-based QST protocol that uses shadow-estimated infidelity as the loss function, named “neural-shadow quantum state tomography” (NSQST). After introducing NNQST and the classical shadow formalism, I will present numerical results on the advantage of NSQST over NNQST at learning the relative phases, NSQST’s noise robustness, and NSQST’s advantage over direct shadow estimation. I will also briefly discuss the future prospects of the protocol with different variational ansatz and randomized measurements, as well as its experimental feasibility.

---

Zoom link https://pitp.zoom.us/j/94167105773?pwd=TXR3TUtwNjV4VFB4SEpvTkhqd29SUT09

Numerical simulations of collision events within the ATLAS experiment have played a pivotal role in shaping the design of future experiments and analyzing ongoing ones. However, the quest for accuracy in describing Large Hadron Collider (LHC) collisions comes at an imposing computational cost, with projections estimating the need for millions of CPU-years annually during the High Luminosity LHC (HL-LHC) run. Simulating a single LHC event with Geant4 currently devours around 1000 CPU seconds, with calorimeter simulations imposing substantial computational demands. To address this challenge, we propose a Quantum-Assisted deep generative model. Our model marries a variational autoencoder (VAE) on the exterior with a Restricted Boltzmann Machine (RBM) in the latent space, delivering enhanced expressiveness compared to conventional VAEs. The RBM nodes and connections are meticulously engineered to enable the use of qubits and couplers on D-Wave's Pegasus Quantum Annealer. We also provide preliminary insights into the requisite infrastructure for large-scale deployment.

---

Zoom link https://pitp.zoom.us/j/97724484247?pwd=Witua1lKcHlrc3JDNHNDWXpHYkVvQT09

The performance of neural networks like large language models (LLMs) is governed by "scaling laws": the error of the network, averaged across the whole dataset, drops as a power law in the number of network parameters and the amount of data the network was trained on. While the mean error drops smoothly and predictably, scaled up LLMs seem to have qualitatively different (emergent) capabilities than smaller versions when one evaluates them at specific tasks. So how does scaling change what neural networks learn? We propose the "quantization model" of neural scaling, where smooth power laws in mean loss are understood as averaging over many small discrete jumps in network performance. Inspired by Max Planck's assumption in 1900 that energy is quantized, we make the assumption that the knowledge or skills that networks must learn are quantized, coming in discrete chunks which we call "quanta". In our model, neural networks can be understand as being implicitly a large number of modules, and scaling simply adds modules to the network. In this talk, I will discuss evidence for and against this hypothesis, its implications for interpretability and for further scaling, and how it fits in with a broader vision for a "science of deep learning".

---

Zoom link https://pitp.zoom.us/j/93886741739?pwd=NzJrcTBNS2xEUUhXajgyak94LzVvdz09