Machine Learning Initiative

The performance of neural networks like large language models (LLMs) is governed by "scaling laws": the error of the network, averaged across the whole dataset, drops as a power law in the number of network parameters and the amount of data the network was trained on. While the mean error drops smoothly and predictably, scaled up LLMs seem to have qualitatively different (emergent) capabilities than smaller versions when one evaluates them at specific tasks. So how does scaling change what neural networks learn? We propose the "quantization model" of neural scaling, where smooth power laws in mean loss are understood as averaging over many small discrete jumps in network performance. Inspired by Max Planck's assumption in 1900 that energy is quantized, we make the assumption that the knowledge or skills that networks must learn are quantized, coming in discrete chunks which we call "quanta". In our model, neural networks can be understand as being implicitly a large number of modules, and scaling simply adds modules to the network. In this talk, I will discuss evidence for and against this hypothesis, its implications for interpretability and for further scaling, and how it fits in with a broader vision for a "science of deep learning".

---

Zoom link https://pitp.zoom.us/j/93886741739?pwd=NzJrcTBNS2xEUUhXajgyak94LzVvdz09

With rapid progress in simulation of strongly interacting quantum Hamiltonians, the challenge in characterizing unknown phases becomes a bottleneck for scientific progress. We demonstrate that a Quantum-Classical hybrid approach (QuCl) of mining the projective snapshots with interpretable classical machine learning, can unveil new signatures of seemingly featureless quantum states. The Kitaev-Heisenberg model on a honeycomb lattice with bond-dependent frustrated interactions presents an ideal system to test QuCl. The model hosts a wealth of quantum spin liquid states: gapped and gapless Z2 spin liquids, and a chiral spin liquid (CSL) phase in a small external magnetic field. Recently, various simulations have found a new intermediate gapless phase (IGP), sandwiched between the CSL and a partially polarized phase, launching a debate over its elusive nature. We reveal signatures of phases in the model by contrasting two phases pairwise using an interpretable neural network, the correlator convolutional neural network (CCNN). We train the CCNN with a labeled collection of sampled projective measurements and reveal signatures of each phase through regularization path analysis. We show that QuCl reproduces known features of established spin liquid phases and ordered phases. Most significantly, we identify a signature motif of the field-induced IGP in the spin channel perpendicular to the field direction, which we interpret as a signature of Friedel oscillations of gapless spinons forming a Fermi surface. Our predictions can guide future experimental searches for U(1) spin liquids.

--

Zoom link: https://pitp.zoom.us/j/94233944575?pwd=OVljLzMrZzlKeUErNHZQRkEzMFRKUT09

Tensor Processing Units are application specific integrated circuits (ASICs) built by Google to run large-scale machine learning (ML) workloads (e.g. AlphaFold). They excel at matrix multiplications, and hence can be repurposed for applications beyond ML. In this talk I will explain how TPUs can be leveraged to run large-scale density matrix renormalization group (DMRG) calculations at unprecedented size and accuracy. DMRG is a powerful tensor network algorithm originally applied to computing ground-states and low-lying excited states of strongly correlated, low-dimensional quantum systems. For certain systems, like one-dimensional gapped or quantum critical Hamiltonians, or small, strongly correlated molecules, it has today become the gold standard method for computing e.g. ground-state properties. Using a TPUv3-pod, we ran large-scale DMRG simulations for a system of 100 spinless fermions, and optimized matrix product state wave functions with a bond dimension of more than 65000 (a parameter space with more than 600 billion parameters). Our results clearly indicate that hardware accelerator platforms like Google's latest TPU versions or NVIDIAs DGX systems are ideally suited to scale tensor network algorithms to sizes that are beyond capabilities of traditional HPC architectures.

Zoom link:  https://pitp.zoom.us/j/99337818378?pwd=SGZvdFFValJQaDNMQ0U1YnJ6NU1FQT09

Polyaromatic hydrocarbons (PAHs) such as acenes have long been studied due to its interesting optical properties and low singlet triplet gaps. Earlier studies have already noticed that use of complete valence active space is imperative to the understanding of its qualitative and quantitative properties. Since complete active space based methods cannot be applied to such large active spaces, we have used density matrix renormalization group (DMRG) based approaches. Further small modification to the PAH topology shows interesting new phases of behaviour in its optical gaps. We have understood the effect of these effects based on spin frustration due to the presence of odd membered rings. In this talk, I will discuss these observations from molecular and model Hamiltonian perspectives.Further developments based on artificial neural network based configuration interaction for strongly correlated systems will also be discussed.5  The similarities between the ANNs and the MPS wavefunctions will be leveraged for 2D systems.

Zoom link:  https://pitp.zoom.us/j/92159136836?pwd=ZFJBcXZ3R3czSUcxcThOci9ueStBZz09

The recent advancement of machine learning provides new opportunities for tackling challenges in quantum science, ranging from condensed matter physics, high energy physics to quantum information science. In this talk, I will first discuss a class of anti-symmetric wave functions based on neural network backflow,  which is efficient for simulating strongly-correlated lattice models and artificial quantum materials. Next, I will talk about recent progress of simulating continuum quantum field theories with neural quantum field state, and lattice gauge theories such as 2+1D quantum electrodynamics with finite density dynamical fermions using gauge symmetric neural networks. I will further discuss neural network representation based on positive-value-operator and phase space measurements for quantum dynamics simulations. Finally, I will present applications of machine learning in quantum control, quantum optimization and quantum machine learning.

Zoom link:  https://pitp.zoom.us/j/93834456412?pwd=R0hxdEpxanFFRnZmTHlqZTBXRi82QT09

Generative modeling via diffusion processes is already a vast field of literature. In this introduction, I will give an entry point to this field by going over the main concepts and deriving the essential results of the area. Thus, by the end of the talk, we would have a minimal pipeline for implementing the generative model. Furthermore, I will outline several alternative ways for learning such models that the community has developed in recent years. These directions bring novel perspectives and new capabilities of generative modeling.

Zoom link:  https://pitp.zoom.us/j/98786491081?pwd=U1cvZzBQT2VUZDl5Ykd0c1lqY29aZz09

Recently, machine learning has become a popular tool to use in fundamental science, including lattice field theory. Here I will report on some recent progress, including the Inverse Renormalisation Group and quantum-field theoretical machine learning, combining insights of lattice field theory and machine learning in a hopefully constructive manner.

Zoom link:  https://pitp.zoom.us/j/95456375462?pwd=WmtZMloyclAyZzBwVEZHQ3gxVnkrUT09

In recent years, machine learning has been successfully used to identify phase transitions and classify phases of matter in a data-driven manner. Neural network (NN)-based approaches are particularly appealing due to the ability of NNs to learn arbitrary functions. However, the larger an NN, the more computational resources are needed to train it, and the more difficult it is to understand its decision making. Thus, we still understand little about the working principle of such machine learning approaches, when they fail or succeed, and how they differ from traditional approaches. In this talk, I will present analytical expressions for the optimal predictions of three popular NN-based methods for detecting phase transitions that rely on solving classification and regression tasks using supervised learning at their core. These predictions are optimal in the sense that they minimize the target loss function. Therefore, in practice, optimal predictive models are well approximated by high-capacity predictive models, such as large NNs after ideal training. I will show that the analytical expressions we have derived provide a deeper understanding of a variety of previous NN-based studies and enable a more efficient numerical routine for detecting phase transitions from data.

Zoom Link: https://pitp.zoom.us/j/91642481966?pwd=alkrWEFFcFBvRlJEbDRBZWV3MFFDUT09

This talk will consider a stochastic multiple-timescale dynamical system modeling a biological neuron. With this model, we will separately uncover the mechanisms underlying two different ways biological neurons encode information with stochastic perturbations: self-induced stochastic resonance (SISR) and inverse stochastic resonance (ISR). We will then show that in the same weak noise limit, SISR and ISR are related through the relative geometric positioning (and stability) of the fixed point and the generic folded singularity of the model’s critical manifold.  This mathematical result could explain the experimental observation where neurons with identical morphological features sometimes encode different information with identical synaptic input. Finally, if time permits, we shall discuss the plausible applications of this result in neuro-biologically inspired machine learning algorithms, particularly reservoir computing based on liquid-state machines.

Zoom link:  https://pitp.zoom.us/j/94345141890?pwd=aTRFM3M0a0xCOEM3aXZjY2hFYzVrQT09

This talk is motivated by the question: why do we put so much effort and investment into quantum computing? A short answer is that we expect quantum advantages for practical problems. To achieve this goal, it is essential to reexamine existing experiments and propose new protocols for future quantum advantage experiments. In 2019, Google published a paper in Nature claiming to have achieved quantum computational advantage, also known as quantum supremacy. In this talk, I will explain how they arrived at their claim and its implications. I will also discuss recent theoretical and numerical developments that challenge this claim and reveal fundamental limitations in their approach. Due to these new developments, it is imperative to design the next generation of experiments. I will briefly mention three potential approaches: efficient verifiable quantum advantage, hardware-efficient fault-tolerance, and quantum algorithms on analog devices, including machine learning and combinatorial optimization.

Zoom Link: https://pitp.zoom.us/j/96945612624?pwd=ckRKMFJqZ0Q0dGtFOU91c1hnMzIzZz09

Recent advances in programmable quantum devices brought to the fore the intriguing possibility of using them to realize and investigate topological quantum spin liquids (QSLs) phase. This new and exciting direction brings about important research questions on how to probe and determine the presence of such exotic, highly entangled phases. In this talk, I will discuss how to construct Z2 QSLs states as the ground state of a static Hamiltonian with only local two-qubit interactions and a transverse field, and demonstrate its realization in the classical limit at the endpoint of quantum annealing protocol, using D-Wave DW-2000Q machine . I will also demonstrate how to probe signatures of Z2 QSLs fractional statistics in quantum simulators via quasiparticle interferometry. At the end, I will show the robustness of this probe against disorders and dephasing -- effects that are generally pervasive in quantum devices nowadays.

Zoom Link: https://pitp.zoom.us/j/99207971920?pwd=N3R3YUJtWXJWVjFQWlRoZ3hYSDlMZz09

We present an approach for representing fermionic quantum many-body states using tensor networks, by introducing a change of basis with local unitary gates obtained via compressing fermionic Gaussian states into quantum circuits. These fermionic Gaussian circuits enable efficient disentangling of low-energy states and entanglement renormalization in matrix product states/operators, significantly reducing bond dimension and improving computational efficiency. As a demonstration, we apply this approach to the 1D single impurity Anderson model through suppression of entanglement in both ground states and time-evolved low-lying excited states. We also explore the use of hierarchical compression to generate Gaussian multi-scale entanglement renormalization ansatz (GMERA) circuits and study their emergent coarse-grained physical models in terms of entanglement properties and suitability for time evolution.

Zoom link:  https://pitp.zoom.us/j/99677586832?pwd=ZGdIVGpac3pWcnhJYjlkNU16Wmlndz09