Artificial Intelligence and Machine Learning

AI and ML for robotics, automated planning, game theory, speech recognition, data visualization, rule-based expert systems and logic programs.

ISR artificial intelligence research on hierarchical task network planning has influenced nearly all subsequent work in this area. We have deep understanding of artificial intelligence planning and the use of mean-field game theory to predict decisions. Our pioneering history in data visualization produced Spotfire, a starfield multidimensional data visualization display tool using dynamic queries and Treemaps, a space-filling method of visualizing large hierarchical collections of quantitative data that gives users the ability to see thousands of data bits in a fixed space that facilitates discovery of patterns, clusters and outliers.  We have expertise in formal methods for description and analysis of concurrent and distributed systems, model checking and abstract interpretation for embedded control and systems biology. Today we are introducing innovations in computer vision and hyperdimensional computing theory for robots, as well as geometric and scientific algorithms for autonomous vehicles, computer graphics, and virtual reality.

Recent publications


Following Instructions by Imagining and Reaching Visual Goals

John Kanu, Eadom Dessalene, Xiaomin Lin, Cornelia Fermüller, Yiannis Aloimonos

A novel robotic agent framework for learning to perform temporally extended tasks using spatial reasoning in a deep reinforcement learning framework, by sequentially imagining visual goals and choosing appropriate actions to fulfill imagined goals.


Order Effects of Measurements in Multi-Agent Hypothesis Testing

Aneesh Raghavan, John Baras

This paper pertains to stochastic multi-agent decision-making problems. The authors revisit the concepts of event-state-operation-structure and relationship of incompatibility from literature, and use them as a tool to study the algebraic structure of a set of events. They consider a multi-agent hypothesis testing problem and show that the set of events forms an ortholattice. They then consider the binary hypothesis testing problem wth finite observation space.

Cooperative Hypothesis Testing by Two Obervers with Asymmetric Information

Aneesh Raghavan, John Baras

This paper pertains to hypothesis testing problems, specifically the problem of collaborative binary hypothesis testing.

Interpretable machine learning models: A physics-based view

Ion Matei, Johan de Kleer, Christoforos Somarakis, Rahul Rai, John Baras

To understand changes in physical systems and facilitate decisions, explaining how model predictions are made is crucial. In this paper the authors use model-based interpretability, where models of physical systems are constructed by composing basic constructs that explain locally how energy is exchanged and transformed.


Event-Triggered Add-on Safety for Connected and Automated Vehicles using Roadside Network Infrastructure

Mohammad Mamduhi, Karl Johansson, Ehsan Hashemi, John Baras

This paper proposes an event-triggered, add-on safety mechanism in a networked vehicular system that can adjust control parameters for timely braking while maintaining maneuverability.


Better Automata through Process Algebra

Rance Cleaveland

This paper shows how the use of Structural Operational Semantics (SOS) inthe style popularized by the process-algebra community can lead to a more succinct and useful construction for building finite automata from regular expressions.


Simulation-based algorithms for Markov decision processes: Monte Carlo tree search from AlphaGo to AlphaZero

Michael Fu

The deep neural networks of AlphaGo and AlphaZero can be traced back to an adaptive multistage sampling (AMS) simulation-based algorithm for Markov decision processed published by HS Chang, Michael C. Fu and Steven I Marcus in Operations Research in 2005. Here, Fu retraces history, talks about the impact of the initial research, and suggests enhancements for the future.

Asian-Pacific Journal of Operational Research


Automatic Shape Optimization of Patient-Specific Tissue Engineered Vascular Grafts for Aortic Coarctation

Xiaolong Liu, Seda Aslan, Rachel Hess, Paige Mass, Laura Olivieri, Yue-Hin Loke, Narutoshi Hibino, Mark Fuge, Axel Krieger

Develops a computational framework for automatically designing optimal shapes of patient-specific TEVGs for aorta surgery.

42nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society

Adaptive Expansion Bayesian Optimization for Unbounded Global Optimization

Wei Chen, Mark Fuge

The authors propose a Bayesian optimization approach that only needs to specify an initial search space that does not necessarily include the global optimum, and expands the search space when necessary.


Reactive Navigation Under Non-Parametric Uncertainty Through Hilbert Space Embedding of Probabilistic Velocity Obstacles

SriSai Naga Jyotish Poonganam, Bharath Gopalakrishnan, Venkata Seetharama Sai Bhargav Kumar Avula, K. Madhava Krishna, Arun Kumar Singh, Dinesh Manocha

A new model predictive control framework that improves reactive navigation for autonomous robots. The framework allows roboticists to compute low cost control inputs while ensuring some upper bound on the risk of collision.

IEEE Robotics and Automation Letters

RoadTrack: Realtime Tracking of Road Agents in Dense and Heterogeneous Environments

Dinesh Manocha, Rohan Chandra, Uttaran Bhattacharya, Aniket Bera, Tanmay Randhavane

The authors' RoadTrack algorithm could help autonomous vehicles navigate dense traffic scenarios. The algorithm uses tracking-by-detection approach to detect vehicles and pedestrians, then predict where they are going.

IROS 2019


The Liar’s Walk: Detecting Deception with Gait and Gesture

Kurt Gray, Tanmay Randhavane, Kyra Kapsaskis, Uttaran Bhattacharya, Aniket Bera, Dinesh Manocha

A data-driven deep neural algorithm for detecting deceptive walking behavior using nonverbal cues like gaits and gestures.

Forecasting Trajectory and Behavior of Road-Agents Using Spectral Clustering in Graph-LSTMs

Rohan Chandra, Tianrui Guan, Srujan Panuganti, Trisha Mittal, Uttaran Bhattacharya, Aniket Bera, Dinesh Manocha

A novel approach for traffic forecasting in urban traffic scenarios using a combination of spectral graph analysis and deep learning.

NeoNav: Improving the Generalization of Visual Navigation via Generating Next Expected Observations

Qiaoyun Wu, Dinesh Manocha, Jun Wang, Kai Xu

The authors improve the cross-target and cross-scene generalization of visual navigation through a learning agent guided by conceiving the next observations it expects to see. A variational Bayesian model, NeoNav, generates the next expected observations (NEO) conditioned on the current observations of the agent and the target view.


Take an Emotion Walk: Perceiving Emotions from Gaits Using Hierarchical Attention Pooling and Affective Mapping

Uttaran Bhattacharya, Christian Roncal, Trisha Mittal, Rohan Chandra,Aniket Bera, Dinesh Manocha

The paper presents an autoencoder-based semi-supervised approach to classify perceived human emotions from walking styles obtained from videos or from motion-captured data and represented as sequences of 3D poses.

Personality-Aware Probabilistic Map for Trajectory Prediction of Pedestrians

Chaochao Li, Pei Lv, Mingliang Xu, Xinyu Wang, Dinesh Manocha, Bing Zhou, Meng Wang

In many applications such as human-robot interaction, autonomous driving or surveillance, it is important to accurately predict pedestrian trajectories for collision-free navigation or abnormal behavior detection. The authors present a novel trajectory prediction algorithm for pedestrians based on a personality-aware probabilistic feature map.

STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaits

Uttaran Bhattacharya, Trisha Mittal, Rohan Chandra, Tanmay Randhavane (UNC), Aniket Bera, and Dinesh Manocha

STEP is a novel classifier network able to classify perceived human emotion from gaits, based on a Spatial Temporal Graph Convolutional Network architecture. Given an RGB video of an individual walking, STEP implicitly exploits the gait features to classify the emotional state of the human into one of four emotions: happy, sad, angry, or neutral. | Watch a video about STEP |

GraphRQI: Classifying Driver Behaviors Using Graph Spectrums

Rohan Chandra, Uttaran Bhattacharya, Trisha Mittal, Xiaoyu Li, Aniket Bera, Dinesh Manocha

The GraphRQI algorithm identifies driver behaviors from road agent trajectories. It is 25 percent more accurate over prior behavior classification algorithms for autonomous vehicles.  | Watch a video about GraphRQI |

Realtime Simulation of Thin-Shell Deformable Materials using CNN-Based Mesh Embedding

Qingyang Tan, Zherong Pan, Lin Gao, and Dinesh Manocha

A new method bridges the gap between mesh embedding and physical simulation for efficient dynamic models of clothes. The key technique is a graph-based convolutional neural network (CNN) defined on meshes with arbitrary topologies, and a new mesh embedding approach based on physics-inspired loss term. After training, the learned simulator runs10–100 times faster and the accuracy is high enough for robot manipulation tasks. | Watch a video about this method |


Using Data Partitions and Stateless Servers to Scale Up Fedora Repositories

Gregory Jansen, Aaron Coburn, Adam Soroka, Richard Marciano

Describes the development and testing of the next-generation Trellis Linked Data Platform with Memento versioning support.


Computational thinking in archival science research and education

William Underwood and Richard Marciano

This paper explores whether the computational thinking practices of mathematicians and scientists in the physical and biological sciences are also the practices of archival scientists. It is argued that these practices are essential elements of an archival science education in preparing students for a professional archival career.

Reframing Digital Curation Practices through a Computational Thinking Framework

Marciano and 20  students in the Digital Curation Innovation Center developed a reframing model for digital curation through computational thinking. Their case study involves adding metadata to non-digital primary records from the WWII TuleLake Japanese American Internment Camp. Their curation methods led to the discovery of new narratives and connections from this data.


Integrating acting, planning, and learning in hierarchical operational models

Sunandita Patra, Amit Kumar, James Mason, Malik Ghallab, Paolo Traverso, Dana Nau

New planning and learning algorithms for Refinement Acting Engine (RAE), which uses hierarchical operational models to perform tasks in dynamically changing environments. 

2020 International Conference on Automated Planning and Scheduling (ICAPS)


APE: Acting and Planning Engine

Sudanita Patra, Malik Ghallab, Dana Nau, Paolo Traverso

An integrated acting and planning system that addresses the consistency problem by using the actor’s operational models both for acting and for planning.


Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy

Ben Shneiderman

Proposes a two-dimensional framework alternative to autonomous AI systems called Human-Centered Artificial Intelligence that clarifies how to design for high levels of human control and high levels of computer automation to increase human performance, understand the situations in which full human control or full computer control are necessary, and avoid the dangers of either excessive human control or excessive computer control.

ACM International Journal of Human-Computer Interaction

ISR artificial intelligence and machine learning news