Event
CDS Invited Lecture: Anuradha Annaswamy, Adaptive Control Intersections with Reinforcement Learning
Friday, December 8, 2023
11:00 a.m.
Online presentation
Control and Dynamical Systems Invited Lecture
Adaptive Control and Intersections with Reinforcement Learning
Anuradha Annaswamy
Active-Adaptive Control Laboratory
Department of Mechanical Engineering
MIT
This is a Zoom online presentation.
https://umd.zoom.us/j/95889318279?pwd=U1FyU3BZZGYwU3lzU3BsanRaazZXdz09 Meeting ID: 958 8931 8279 Passcode: 404644
Abstract
Adaptive control and reinforcement learning based control are two different methods that are both commonly employed for the control of uncertain systems. Historically, adaptive control has excelled at real-time control of systems with specific model structures through adaptive rules that learn the underlying parameters while providing strict guarantees on stability, asymptotic performance, and learning. Reinforcement learning methods are applicable to a broad class of systems and are able to produce near-optimal policies for highly complex control tasks. This is often enabled by significant offline training via simulation or the collection of large input-state datasets. In both methods, the main approach used for updating the parameters is based on a gradient descent-like algorithm. Related tools of analysis, convergence, and robustness in both fields have a tremendous amount of similarity.
This talk will examine the similarities and interconnections between adaptive control and reinforcement learning-based control. Concepts in stability, performance, and learning, common to both methods will be discussed. Building on the similarities in update laws and common concepts, new intersections and opportunities for improved algorithm analysis will be explored. Two specific examples of dynamic systems are used to illustrate the details of the two methods, their advantages, and their deficiencies. We will explore how these methods can be leveraged and integrated to lead to provably correct methods for learning in real-time with guaranteed fast convergence. Examples will be drawn from a range of engineering applications.
Biography
Dr. Anuradha Annaswamy is Founder and Director of the Active-Adaptive Control Laboratory in the Department of Mechanical Engineering at MIT. Her research interests span adaptive control theory and its applications to several engineering systems including to aerospace, automotive, propulsion, and energy systems, cyber-enabled energy grids, and urban mobility. She has received best paper awards (Axelby; CSM), Distinguished Member and Distinguished Lecturer awards from the IEEE Control Systems Society (CSS), Best Paper award from IFAC for Annual Reviews in Control (2021-23), and a Presidential Young Investigator award from NSF. She is a Fellow of IEEE and IFAC. She is the recipient of the Distinguished Alumni award from Indian Institute of Science for 2021. She was a Hans-Fischer Senior Fellow, Institute for Advanced Study, Technical University of Munich, Germany during the year 2008-09. She received Donald Julius Groen Prize, from the Institution of Mechanical Engineers, UK in 2008.
Annaswamy is the author of a graduate textbook on adaptive control and several journal and conference publications, and co-editor of two vision documents on smart grids, two editions of the Impact of Control Technology report, and the 2023 CSS report “Control for Societal-scale Challenges: Road Map 2030”. She is also a coauthor of a 2021 National Academy of Sciences, Engineering, and Medicine (NASEM) Committee report on the Future of Electric Power in the United States, and a 2023 NASEM report on the Role of Net-metering in the Evolving Electricity System. She served as the President of CSS in 2020. She is a Faculty Lead in the Electric Power Systems workstream in the MIT Future Energy Systems Center.
