Talks

 Reinforcement learning (RL) has been the driver behind many of the most significant advances in artificial intelligence over the past decade---ranging from achieving superhuman performance in complex games like Go and starcraft to applications in autonomous driving, robotics, and economic simulations. RL is even playing a crucial role in the fine tuning of large language models and the training of AI agents more broadly. Many of these problems, however, are fundamentally multi-agent in nature: an agent's success is inextricably linked to the decisions of others. Despite these empirical successes and a wealth of research on "single-agent" RL and its variants, multi-agent reinforcement learning (MARL) remains relatively under-explored theoretically with the presence of multiple learning agents giving rise to a unique set of challenges for algorithm design and analysis. 

This tutorial will give an overview of the research landscape in MARL, aiming to highlight the core theoretical principles that enable agents to learn and adapt in the presence of others. Using the formal framework of Markov games and building on a foundation in game theory, we will explore the different solution concepts and algorithms in the field. The discussion will explore the inherent inefficiencies of classical algorithms like fictitious play and policy gradient algorithms and build toward the principles underpinning modern, provably efficient learning methods for large games (i.e., algorithms that make use of function approximation like deep neural networks).  Ultimately, we will identify key open problems and promising new  research directions for the future of multi-agent learning.

The existence of a population of massive quiescent galaxies with little to no star formation poses a challenge to our understanding of galaxy evolution. The physical process that quenched the star formation in these galaxies is debated, but the most popular possibility is that feedback from supermassive black holes lifts or heats the gas that would otherwise be used to form stars. In this paper, we evaluate this idea in two ways. First, we compare the cumulative growth in the cosmic inventory of the total stellar mass in quiescent galaxies to the corresponding growth in the amount of kinetic energy carried by radio jets. We find that these two inventories are remarkably well-synchronized, with about 50% of the total amounts being created in the epoch from z ≈ 1 to 2. We also show that these agree extremely well with the corresponding growth in the cumulative number of major mergers that result in massive (>10^11 M_ ʘ) galaxies. We therefore argue that major mergers trigger the radio jets and also transform the galaxies from disks to spheroids. Second, we evaluate the total amount of kinetic energy delivered by jets and compare it to the baryonic binding energy of the galaxies. We find the jet kinetic energy is more than sufficient to quench star formation, and the quenching process should be more effective in more massive galaxies. We show that these results are quantitatively consistent with recent measurements of the Sunyaev–Zel'dovich effect seen in massive galaxies at z ≈ 1.

The effects of baryonic feedback on matter power spectrum are uncertain. Upcoming large-scale structure surveys require percent-level constraints on the impact of baryonic feedback effects on the small-scale ($k \gtrsim 1\,h\,$Mpc$^{-1}$) matter power spectrum to fully exploit weak lensing data. The sightline-to-sightline variance in the fast radio bursts (FRBs) dispersion measure (DM) correlates with the strength of baryonic feedback and offers unique sensitivity at scales upto $k \sim 100\,h\,$Mpc$^{-1}$. We analytically compute the variance in FRB DMs using the electron power spectrum, which is modeled as a function of cosmological and feedback parameters in IllustrisTNG suite of simulations in CAMELS project. We demonstrate its efficacy in capturing baryonic feedback effects across several simulation suites, including SIMBA and Astrid. We show that with 10,000 FRBs, the suppression of the matter power spectrum can be constrained to percent-level precision at large scales (k < 1 h/Mpc) and ~10% precision at small scales (k > 10 h/Mpc). Insights into the impact of baryons on the small-scale matter power spectrum gained from FRBs can be leveraged to mitigate baryonic uncertainties in cosmic shear analyses.

Fast Radio Bursts (FRBs) are powerful probes of diffuse ionized baryons, offering unique insights into the cosmic ecosystems from the circumgalactic medium (CGM) to the intergalactic medium (IGM). Utilizing simulation suites from the CAMELS project—IllustrisTNG, SIMBA, and Astrid—we analyze FRB dispersion measures (DMs) across models with varying cosmological and astrophysical parameters. Our analysis shows that DM radial profiles around the CGM are highly sensitive to baryonic effects, with strong ejective feedback causing baryon spread in and around halos. On larger scales, we introduce "baryon spread" as a robust measure of baryonic impact on the matter power spectrum. Our study reveals a strong correlation between FRB statistics, particularly the F-parameter, and baryon spread in CAMELS simulations, independent of subgrid galaxy formation models. This correlation offers a novel pathway for using FRBs to correct for baryonic effects in ongoing and upcoming cosmological surveys, such as DESI, Euclid, Roman, and Rubin. With large FRB samples, our findings highlight the pivotal role of FRBs in bridging astrophysics and cosmology, offering new constraints on the CGM and enhancing the power of next-generation cosmological surveys.

Fast radio bursts (FRBs) are dispersed and scattered by plasma density fluctuations along the line-of-sight, making them sensitive probes of diffuse ionized gas across interstellar, circumgalactic, and intergalactic media. In this talk I will discuss how FRB propagation effects are unveiling cosmic plasmas at extremely small (sub-au) spatial scales in both interstellar and circumgalactic media, and how they may be used in tandem with quasars to constrain the turbulent dynamics of ionized gas in these environments.

The intergalactic medium gas is usually believed to simply trace the cosmic web structures traced by dark matter, but AGN feedback (especially jets) will probably modify this simple picture especially the IGM and CGM baryon budgets. I will talk primarily about the combining spectroscopic galaxy surveys with FRBs to probe the IGM and CGM balance, which is being implemented in the FLIMFLAM survey. I will highlight the role of Subaru PFS, the world's most powerful multiobject spectrograph, in efficiently mapping the cosmic web out to high redshift, and new applications of the field level inference technique towards this problem.

The cosmic web in the distant universe is generally mapped by observing the positions of galaxies. Recently a new perspective has been gained by mapping the diffuse gas in the intergalactic medium (IGM). The Lyman-Alpha Tomography IMACS Survey (LATIS) has now produced 3D maps of the IGM at redshift z~2.5, covering a volume of about 10^7 Mpc^3. I will present the LATIS maps and discuss how they inform our understanding of the interplay between early galaxies and their large-scale environments.

Remarkable parallels exist between observations of diffuse baryons and their connections to galaxies in the high redshift universe compared to z~0 ... in spite of apparently vast differences between the properties of the galaxies themselves and the nature and intensity of the feedback processes operating within them. I will attempt to draw connections between "contemporaneous" observations of galaxy-scale feedback in the adolescent universe with observable signatures "today", and attempt to distinguish between evolutionary sequences and possible coincidences that need further investigation.

Recent observations of the CGM reveal that it is extended, multiphase, and ubiquitous, detected around star forming and quiescent galaxies. However, many questions remain open - how much gas is out there, what are its thermal properties, spatial distribution, and morphology? These are linked to the properties of gas accretion onto galaxies, star formation, and feedback processes, and are crucial to our holistic understanding of galactic ecosystems. I will present the multiphase CGM modeling framework I developed with collaborators and showcase examples of its application to a wide range of absorption measurements, constraining the CGM mass, thermodynamics, energetics, and cool gas cloud sizes. I will also demonstrate how predictions from these models can be used to test them with upcoming and future multi-wavelength observations.