Machine Learning Initiative

Despite their simple intuition, convolutions are more tedious to analyze than dense layers, which complicates the transfer of theoretical and algorithmic ideas. We provide a simplifying perspective onto convolutions through tensor networks (TNs) which allow reasoning about the underlying tensor multiplications by drawing diagrams, and manipulating them to perform function transformations and sub-tensor access. We demonstrate this expressive power by deriving the diagrams of various autodiff operations and popular approximations of second-order information with full hyper-parameter support, batching, channel groups, and generalization to arbitrary convolution dimensions. Further, we provide convolution-specific transformations based on the connectivity pattern which allow to re-wire and simplify diagrams before evaluation. Finally, we probe computational performance, relying on established machinery for efficient TN contraction. Our TN implementation speeds up a recently-proposed KFAC variant up to 4.5x and enables new hardware-efficient tensor dropout for approximate backpropagation.

---

Zoom link https://pitp.zoom.us/j/99090845943?pwd=NHBNVTNnbDNSOGNSVzNGS21xcllFdz09

In quantum metrology, one of the major applications of quantum technologies, the ultimate precision of estimating an unknown parameter is often stated in terms of the Cramér-Rao bound. Yet, the latter is no longer guaranteed to carry an operational meaning in the regime where few measurement samples are obtained. We instead propose to quantify the quality of a metrology protocol by the probability of obtaining an estimate with a given accuracy. This approach, which we refer to as probably approximately correct (PAC) metrology, ensures operational significance in the finite-sample regime. The accuracy guarantees hold for any value of the unknown parameter, unlike the Cramér-Rao bound which assumes it is approximately known. We establish a strong connection to multi-hypothesis testing with quantum states, which allows us to derive an analogue of the Cramér-Rao bound which contains explicit corrections relevant to the finite-sample regime. We further study the asymptotic behavior of the success probability of the estimation procedure for many copies of the state and apply our framework to the example task of phase estimation with an ensemble of spin-1/2 particles. Overall, our operational approach allows the study of quantum metrology in the finite-sample regime and opens up a plethora of new avenues for research at the interface of quantum information theory and quantum metrology. TL;DR: In this talk, I will motivate why the Cramér-Rao bound might not always be the tool of choice to quantify the ultimate precision attainable in a quantum metrology task and give a (hopefully) intuitive introduction of how we propose to instead quantify it in a way that is valid in the single- and few-shot settings. We will together unearth a strong connection to quantum multi-hypothesis testing and conclude that there are many exiting and fundamental open questions in single-shot metrology!

---

Zoom link https://pitp.zoom.us/j/92247273192?pwd=ZkprOFZ0eEdQYjJDY1hneFNLckFDZz09

Quantum state tomography (QST) is the art of reconstructing an unknown quantum state through measurements. It is a key primitive for developing quantum technologies. Neural network quantum state tomography (NNQST), which aims to reconstruct the quantum state via a neural network ansatz, is often implemented via a basis-dependent cross-entropy loss function. State-of-the-art implementations of NNQST are often restricted to characterizing a particular subclass of states, to avoid an exponential growth in the number of required measurement settings. In this talk, I will discuss an alternative neural-network-based QST protocol that uses shadow-estimated infidelity as the loss function, named “neural-shadow quantum state tomography” (NSQST). After introducing NNQST and the classical shadow formalism, I will present numerical results on the advantage of NSQST over NNQST at learning the relative phases, NSQST’s noise robustness, and NSQST’s advantage over direct shadow estimation. I will also briefly discuss the future prospects of the protocol with different variational ansatz and randomized measurements, as well as its experimental feasibility.

---

Zoom link https://pitp.zoom.us/j/94167105773?pwd=TXR3TUtwNjV4VFB4SEpvTkhqd29SUT09

Numerical simulations of collision events within the ATLAS experiment have played a pivotal role in shaping the design of future experiments and analyzing ongoing ones. However, the quest for accuracy in describing Large Hadron Collider (LHC) collisions comes at an imposing computational cost, with projections estimating the need for millions of CPU-years annually during the High Luminosity LHC (HL-LHC) run. Simulating a single LHC event with Geant4 currently devours around 1000 CPU seconds, with calorimeter simulations imposing substantial computational demands. To address this challenge, we propose a Quantum-Assisted deep generative model. Our model marries a variational autoencoder (VAE) on the exterior with a Restricted Boltzmann Machine (RBM) in the latent space, delivering enhanced expressiveness compared to conventional VAEs. The RBM nodes and connections are meticulously engineered to enable the use of qubits and couplers on D-Wave's Pegasus Quantum Annealer. We also provide preliminary insights into the requisite infrastructure for large-scale deployment.

---

Zoom link https://pitp.zoom.us/j/97724484247?pwd=Witua1lKcHlrc3JDNHNDWXpHYkVvQT09

The performance of neural networks like large language models (LLMs) is governed by "scaling laws": the error of the network, averaged across the whole dataset, drops as a power law in the number of network parameters and the amount of data the network was trained on. While the mean error drops smoothly and predictably, scaled up LLMs seem to have qualitatively different (emergent) capabilities than smaller versions when one evaluates them at specific tasks. So how does scaling change what neural networks learn? We propose the "quantization model" of neural scaling, where smooth power laws in mean loss are understood as averaging over many small discrete jumps in network performance. Inspired by Max Planck's assumption in 1900 that energy is quantized, we make the assumption that the knowledge or skills that networks must learn are quantized, coming in discrete chunks which we call "quanta". In our model, neural networks can be understand as being implicitly a large number of modules, and scaling simply adds modules to the network. In this talk, I will discuss evidence for and against this hypothesis, its implications for interpretability and for further scaling, and how it fits in with a broader vision for a "science of deep learning".

---

Zoom link https://pitp.zoom.us/j/93886741739?pwd=NzJrcTBNS2xEUUhXajgyak94LzVvdz09

With rapid progress in simulation of strongly interacting quantum Hamiltonians, the challenge in characterizing unknown phases becomes a bottleneck for scientific progress. We demonstrate that a Quantum-Classical hybrid approach (QuCl) of mining the projective snapshots with interpretable classical machine learning, can unveil new signatures of seemingly featureless quantum states. The Kitaev-Heisenberg model on a honeycomb lattice with bond-dependent frustrated interactions presents an ideal system to test QuCl. The model hosts a wealth of quantum spin liquid states: gapped and gapless Z2 spin liquids, and a chiral spin liquid (CSL) phase in a small external magnetic field. Recently, various simulations have found a new intermediate gapless phase (IGP), sandwiched between the CSL and a partially polarized phase, launching a debate over its elusive nature. We reveal signatures of phases in the model by contrasting two phases pairwise using an interpretable neural network, the correlator convolutional neural network (CCNN). We train the CCNN with a labeled collection of sampled projective measurements and reveal signatures of each phase through regularization path analysis. We show that QuCl reproduces known features of established spin liquid phases and ordered phases. Most significantly, we identify a signature motif of the field-induced IGP in the spin channel perpendicular to the field direction, which we interpret as a signature of Friedel oscillations of gapless spinons forming a Fermi surface. Our predictions can guide future experimental searches for U(1) spin liquids.

--

Zoom link: https://pitp.zoom.us/j/94233944575?pwd=OVljLzMrZzlKeUErNHZQRkEzMFRKUT09

Tensor Processing Units are application specific integrated circuits (ASICs) built by Google to run large-scale machine learning (ML) workloads (e.g. AlphaFold). They excel at matrix multiplications, and hence can be repurposed for applications beyond ML. In this talk I will explain how TPUs can be leveraged to run large-scale density matrix renormalization group (DMRG) calculations at unprecedented size and accuracy. DMRG is a powerful tensor network algorithm originally applied to computing ground-states and low-lying excited states of strongly correlated, low-dimensional quantum systems. For certain systems, like one-dimensional gapped or quantum critical Hamiltonians, or small, strongly correlated molecules, it has today become the gold standard method for computing e.g. ground-state properties. Using a TPUv3-pod, we ran large-scale DMRG simulations for a system of 100 spinless fermions, and optimized matrix product state wave functions with a bond dimension of more than 65000 (a parameter space with more than 600 billion parameters). Our results clearly indicate that hardware accelerator platforms like Google's latest TPU versions or NVIDIAs DGX systems are ideally suited to scale tensor network algorithms to sizes that are beyond capabilities of traditional HPC architectures.

Zoom link:  https://pitp.zoom.us/j/99337818378?pwd=SGZvdFFValJQaDNMQ0U1YnJ6NU1FQT09

Polyaromatic hydrocarbons (PAHs) such as acenes have long been studied due to its interesting optical properties and low singlet triplet gaps. Earlier studies have already noticed that use of complete valence active space is imperative to the understanding of its qualitative and quantitative properties. Since complete active space based methods cannot be applied to such large active spaces, we have used density matrix renormalization group (DMRG) based approaches. Further small modification to the PAH topology shows interesting new phases of behaviour in its optical gaps. We have understood the effect of these effects based on spin frustration due to the presence of odd membered rings. In this talk, I will discuss these observations from molecular and model Hamiltonian perspectives.Further developments based on artificial neural network based configuration interaction for strongly correlated systems will also be discussed.5  The similarities between the ANNs and the MPS wavefunctions will be leveraged for 2D systems.

Zoom link:  https://pitp.zoom.us/j/92159136836?pwd=ZFJBcXZ3R3czSUcxcThOci9ueStBZz09

The recent advancement of machine learning provides new opportunities for tackling challenges in quantum science, ranging from condensed matter physics, high energy physics to quantum information science. In this talk, I will first discuss a class of anti-symmetric wave functions based on neural network backflow,  which is efficient for simulating strongly-correlated lattice models and artificial quantum materials. Next, I will talk about recent progress of simulating continuum quantum field theories with neural quantum field state, and lattice gauge theories such as 2+1D quantum electrodynamics with finite density dynamical fermions using gauge symmetric neural networks. I will further discuss neural network representation based on positive-value-operator and phase space measurements for quantum dynamics simulations. Finally, I will present applications of machine learning in quantum control, quantum optimization and quantum machine learning.

Zoom link:  https://pitp.zoom.us/j/93834456412?pwd=R0hxdEpxanFFRnZmTHlqZTBXRi82QT09

Generative modeling via diffusion processes is already a vast field of literature. In this introduction, I will give an entry point to this field by going over the main concepts and deriving the essential results of the area. Thus, by the end of the talk, we would have a minimal pipeline for implementing the generative model. Furthermore, I will outline several alternative ways for learning such models that the community has developed in recent years. These directions bring novel perspectives and new capabilities of generative modeling.

Zoom link:  https://pitp.zoom.us/j/98786491081?pwd=U1cvZzBQT2VUZDl5Ykd0c1lqY29aZz09

Recently, machine learning has become a popular tool to use in fundamental science, including lattice field theory. Here I will report on some recent progress, including the Inverse Renormalisation Group and quantum-field theoretical machine learning, combining insights of lattice field theory and machine learning in a hopefully constructive manner.

Zoom link:  https://pitp.zoom.us/j/95456375462?pwd=WmtZMloyclAyZzBwVEZHQ3gxVnkrUT09

In recent years, machine learning has been successfully used to identify phase transitions and classify phases of matter in a data-driven manner. Neural network (NN)-based approaches are particularly appealing due to the ability of NNs to learn arbitrary functions. However, the larger an NN, the more computational resources are needed to train it, and the more difficult it is to understand its decision making. Thus, we still understand little about the working principle of such machine learning approaches, when they fail or succeed, and how they differ from traditional approaches. In this talk, I will present analytical expressions for the optimal predictions of three popular NN-based methods for detecting phase transitions that rely on solving classification and regression tasks using supervised learning at their core. These predictions are optimal in the sense that they minimize the target loss function. Therefore, in practice, optimal predictive models are well approximated by high-capacity predictive models, such as large NNs after ideal training. I will show that the analytical expressions we have derived provide a deeper understanding of a variety of previous NN-based studies and enable a more efficient numerical routine for detecting phase transitions from data.

Zoom Link: https://pitp.zoom.us/j/91642481966?pwd=alkrWEFFcFBvRlJEbDRBZWV3MFFDUT09