Nau, Gelfand, Goldstein part of MURI developing the potential of mean-field game theory

Professor Dana Nau (CS/ISR), Professor Michele Gelfand (Psychology), and Assistant Professor Tom Goldstein (CS) are part of a Multidisciplinary University Research Initiative (MURI), one of 24 awarded this year by the Department of Defense. MURIs provide funding for up to five years to academic institutions for multidisciplinary basic research, subject to satisfactory research progress and the availability of funds. Together, the 2018 MURIs total $169 million in funding.

The Maryland researchers are part of “Innovations in Mean-Field Game Theory for Scalable Computation and Diverse Applications,” an Air Force Office of Scientific Research MURI led by Principal Investigator Stanley Osher of the University of California, Los Angeles, and his colleagues Wilfrid Gangbo and Wotao Yin (all mathematics professors). Two other universities also are part of the MURI: the University of Houston, where Maryland alumnus and ECE Professor Zhu Han (EE Ph.D. 2003) is the researcher; and Princeton University, with researcher H. Vincent Poor (EE), the Michael Henry Strater University Professor of Electrical Engineering.

About the research project

Mean field games (MFGs) are used to study strategic decision making in large populations where individual players interact via certain mean-field quantities. They have recently gained enormous popularity as powerful research tools with applications ranging from tracing quantum mechanics to biodiversity ecology and the evolution of biological systems. MFGs also have the potential to make a great impact on the evolution of social cooperation, macroeconomics, stock markets and wealth distribution, among other fields.

In general, this MURI project aims to make real progress in developing the potential of MFGs through a coordinated effort based on building an overarching mathematical framework, developing accurate modeling in diverse applications, and creating a set of algorithmic tools that support fast simulation, prediction and inverse design. The team of eight experts encompasses the fields of pure math/partial differential equations (Gangbo), applied math/numerical analysis (Osher), optimization (Yin), optimization/machine learning (Goldstein), psychology (Gelfand), artificial intelligence and game theory (Nau) and electrical engineering (Han and Poor).

About the Maryland portion of the research

Nau, Gelfand and their colleagues have been integrating research on evolutionary game theory with cross-cultural psychology for the past decade, publishing in journals such as the Proceedings of the Royal Society B, Nature Scientific Reports, AAMAS, and Organizational Behavior and Human Decision Processes.

In their subproject, “Social Behavior Analysis and Control Based on MFG Framework,” the Maryland researchers will use real-world data to look at ways MFG theory could be used to analyze and validate models of human social norm dynamics.

The nature of human interaction and influence is vastly different around the world. Some cultures, such as Japan, South Korea and countries in the Middle East are considered “tight,” meaning that there are strong social norms, a high degree of adherence and high punishment towards norm violators. Other cultures, such as Australia, New Zealand and the Netherlands, are “loose,” having more tolerance for deviance.

Recently, social uprisings and turmoil around the world have put a spotlight on the importance of understanding how norms are maintained, and how they change. In particular, it is important to better understand how participating in social networks is affecting social norms. These networks have come to influence and occupy people’s daily lives, and hold immense power to foment collective action leading to political change.

Researchers seeking to analyze social networks’ effects on public opinion and action have been limited to using existing, highly simplified models that are limited in their ability to make rigorous quantitative predictions. There is a great need for an analysis framework that can deal with large populations of individuals and accurately reflect the essential nature of dynamical interactions of interest, yet perform such analysis in a simple, computational and mathematical way.

The Maryland researchers believe MFG theory has the potential to accurately demonstrate and predict how social norms change. Since MGF theory decouples a large dimension problem into local decision-making problems of a generic player, it could be a powerful tool for modeling complicated social behaviors.

The researchers will validate and tune the results of theoretical MFG models using real-world data. One example could be the differences in the adoption of healthcare practices in the United States, which vary by state and could be related to the “tightness” or “looseness” of social norms in particular states. Another example could be to analyze Twitter data from the 2016 U.S. presidential election.

Goldstein will work with Nau and the UCLA research team to focus on how artificial intelligence (AI) systems learn by interacting with the world around them.

In the field of “reinforcement learning,” AI agents learn by exploring their environment, building a model of their surroundings, then updating their behavior. Sometimes the environment in which these systems are trained contains other AI agents as well; these agents can improve their skills by competing with each other. For example, an environment could be set up where computer systems compete against each other at a game, and the winning agent earns a reward at the end of each game. By doing this, AI systems can be taught to perform well at the game simply by competing against one another, without relying on expert human knowledge from a trainer.

Unfortunately, the dynamics of these kinds of interactions can sometimes lead to unexpected results. For example, in the game-playing scenario above, the agents might learn to play poorly so that each game ends quickly and a larger number of payouts are distributed in a shorter amount of time. Or perhaps a scenario arises when the agents are at very different skill levels and become unable to learn from one another.

By using ideas from mean-field game theory, Goldstein hopes to understand the dynamics of these kinds of reinforcement learning scenarios, developing a theoretical and practical understanding of how to set up productive training environments, and what mathematical properties of these environments explain bad outcomes.

About the Department of Defense’s 2018 MURI awards

In 2018, the U.S. Department of Defense is issuing 24 awards totaling $169 million to academic institutions to perform multidisciplinary basic research. This is a six million dollar increase above 2017’s total. The awards are for up to a five year period, subject to satisfactory research progress and the availability of funds.

“The Multidisciplinary University Research Initiative program, or MURI, supports research by funding teams of investigators that include more than one traditional science and engineering discipline in order to accelerate the research progress,” said Dale Ormond, Principal Director for Research, in the Office of the Under Secretary of Defense for Research and Engineering. According to Ormond, most of the program’s efforts involve researchers from multiple academic institutions and academic departments. “MURI awards also support the education and training of graduate students in cutting-edge research areas, Ormond stated.

The highly competitive MURI program complements other DoD basic research initiatives that support traditional, three year, single-investigator university research grants. By supporting multidisciplinary teams with larger, longer awards in carefully chosen and relevant research topics, DoD and the Services enhance the potential for significant and sustained advancement of research in critical areas of importance to National Security and the DoD’s mission.

The Army Research Office, the Air Force Office of Scientific Research, and the Office of Naval Research solicited proposals in 24 topic areas important to the DoD and the Services. In response to the initial solicitation the department received 436 white papers. After two rounds of merit-based reviews, a panel of DoD scientists and experts narrowed the proposals to 100 from which the 24 final awards were selected. Based on the proposals selected in the competition, 64 U.S. academic institutions are expected to participate in the 2018 MURI program. In addition, this year, for the first time, Australia has funded research teams to work together with two U.S. MURI teams. This cooperation is in line with the Secretary of Defense’s direction to strengthen our alliances, and collaborate whenever and wherever possible.

Over the past 30 years, DoD’s MURI program resulted in significant capabilities for our military forces and opened up entirely new lines of research. Notable examples include foundations in the fabrication of nanoscale and microscale structures by the processes of self-assembled materials (SAM) and microcontact printing, the integration of vision algorithms with sensors to create low-power, low-latency, compact adaptive vision systems, and advances in fully optical data control and switching. These and other important technological advances from the MURI program have impact on current and future military capabilities.

Published April 20, 2018