Talks

In this series of talks, I will introduce basic elements of generative modeling with diffusion and flow models from first principles. This includes a short introduction to stochastic calculus, ordinary differential equations, evolution of probability measures, Fokker Planck equation, and the continuity equation. We will then apply these ideas to describe training and inference algorithms for diffusion models.

Reinforcement learning (RL) has been the driver behind many of the most significant advances in artificial intelligence over the past decade---ranging from achieving superhuman performance in complex games like Go and starcraft to applications in autonomous driving, robotics, and economic simulations. RL is even playing a crucial role in the fine tuning of large language models and the training of AI agents more broadly. Many of these problems, however, are fundamentally multi-agent in nature: an agent's success is inextricably linked to the decisions of others. Despite these empirical successes and a wealth of research on "single-agent" RL and its variants, multi-agent reinforcement learning (MARL) remains relatively under-explored theoretically with the presence of multiple learning agents giving rise to a unique set of challenges for algorithm design and analysis. 

This tutorial will give an overview of the research landscape in MARL, aiming to highlight the core theoretical principles that enable agents to learn and adapt in the presence of others. Using the formal framework of Markov games and building on a foundation in game theory, we will explore the different solution concepts and algorithms in the field. The discussion will explore the inherent inefficiencies of classical algorithms like fictitious play and policy gradient algorithms and build toward the principles underpinning modern, provably efficient learning methods for large games (i.e., algorithms that make use of function approximation like deep neural networks).  Ultimately, we will identify key open problems and promising new  research directions for the future of multi-agent learning.

The course will cover the basics of reinforcement learning theory. We will start by implementing simple gradient-based algorithms in PyTorch and using them to solve standard control problems like CartPole and the Atari 2600 game Pong. Along the way, we will explore how to optimize both the sample complexity (the number of interactions with the environment) and the computational complexity (GPU hours) needed to learn an optimal policy.

Lecture notes, and setup instructions - https://gomahajan.github.io/icts/rlbootcamp.html 

Optimal transport studies the problem of rearranging one distribution into another while minimizing an associated cost. The past decade has witnessed tremendous progress in our understanding of the computational, methodological and statistical aspects of optimal transport (OT). Recent interest in OT has blossomed due to its close connections with diffusion models.

I will introduce the mathematical framework of OT, and then quickly transition to studying how well various objects in the OT framework (OT distances, and OT maps) can be estimated from samples of the underlying distributions.

Reinforcement learning (RL) has been the driver behind many of the most significant advances in artificial intelligence over the past decade---ranging from achieving superhuman performance in complex games like Go and starcraft to applications in autonomous driving, robotics, and economic simulations. RL is even playing a crucial role in the fine tuning of large language models and the training of AI agents more broadly. Many of these problems, however, are fundamentally multi-agent in nature: an agent's success is inextricably linked to the decisions of others. Despite these empirical successes and a wealth of research on "single-agent" RL and its variants, multi-agent reinforcement learning (MARL) remains relatively under-explored theoretically with the presence of multiple learning agents giving rise to a unique set of challenges for algorithm design and analysis. 

This tutorial will give an overview of the research landscape in MARL, aiming to highlight the core theoretical principles that enable agents to learn and adapt in the presence of others. Using the formal framework of Markov games and building on a foundation in game theory, we will explore the different solution concepts and algorithms in the field. The discussion will explore the inherent inefficiencies of classical algorithms like fictitious play and policy gradient algorithms and build toward the principles underpinning modern, provably efficient learning methods for large games (i.e., algorithms that make use of function approximation like deep neural networks).  Ultimately, we will identify key open problems and promising new  research directions for the future of multi-agent learning.

Reinforcement learning (RL) has been the driver behind many of the most significant advances in artificial intelligence over the past decade---ranging from achieving superhuman performance in complex games like Go and starcraft to applications in autonomous driving, robotics, and economic simulations. RL is even playing a crucial role in the fine tuning of large language models and the training of AI agents more broadly. Many of these problems, however, are fundamentally multi-agent in nature: an agent's success is inextricably linked to the decisions of others. Despite these empirical successes and a wealth of research on "single-agent" RL and its variants, multi-agent reinforcement learning (MARL) remains relatively under-explored theoretically with the presence of multiple learning agents giving rise to a unique set of challenges for algorithm design and analysis. 

This tutorial will give an overview of the research landscape in MARL, aiming to highlight the core theoretical principles that enable agents to learn and adapt in the presence of others. Using the formal framework of Markov games and building on a foundation in game theory, we will explore the different solution concepts and algorithms in the field. The discussion will explore the inherent inefficiencies of classical algorithms like fictitious play and policy gradient algorithms and build toward the principles underpinning modern, provably efficient learning methods for large games (i.e., algorithms that make use of function approximation like deep neural networks).  Ultimately, we will identify key open problems and promising new  research directions for the future of multi-agent learning.

The course will cover the basics of reinforcement learning theory. We will start by implementing simple gradient-based algorithms in PyTorch and using them to solve standard control problems like CartPole and the Atari 2600 game Pong. Along the way, we will explore how to optimize both the sample complexity (the number of interactions with the environment) and the computational complexity (GPU hours) needed to learn an optimal policy.

Lecture notes, and setup instructions - https://gomahajan.github.io/icts/rlbootcamp.html 

 Reinforcement learning (RL) has been the driver behind many of the most significant advances in artificial intelligence over the past decade---ranging from achieving superhuman performance in complex games like Go and starcraft to applications in autonomous driving, robotics, and economic simulations. RL is even playing a crucial role in the fine tuning of large language models and the training of AI agents more broadly. Many of these problems, however, are fundamentally multi-agent in nature: an agent's success is inextricably linked to the decisions of others. Despite these empirical successes and a wealth of research on "single-agent" RL and its variants, multi-agent reinforcement learning (MARL) remains relatively under-explored theoretically with the presence of multiple learning agents giving rise to a unique set of challenges for algorithm design and analysis. 

This tutorial will give an overview of the research landscape in MARL, aiming to highlight the core theoretical principles that enable agents to learn and adapt in the presence of others. Using the formal framework of Markov games and building on a foundation in game theory, we will explore the different solution concepts and algorithms in the field. The discussion will explore the inherent inefficiencies of classical algorithms like fictitious play and policy gradient algorithms and build toward the principles underpinning modern, provably efficient learning methods for large games (i.e., algorithms that make use of function approximation like deep neural networks).  Ultimately, we will identify key open problems and promising new  research directions for the future of multi-agent learning.