Robotics and autonomy

Bio-inspired robots; medical robots; collaborative, cooperative and networked robots; robotics for extreme environments; unmanned air and water vehicles; AI and computer vision for robots

Early in its existence, ISR developed motion description languages and designed and fabricated a modular dexterous hand. More recently, ISR has contributed to advances in underwater robots, micro air vehicles and micro robots; and introduced innovations in computer vision and hyperdimensional computing theory. ISR faculty have advanced flocking and swarming theory, control and design for robotic groups and swarms. A multi-part, in-mold assembly process that reduces process cost and enables robotic design possibilities was pioneered by ISR. ISR founded the Maryland Robotics Center, a thriving research and educational leader, in 2010. Most recently, ISR faculty and students have been at the forefront of the autonomy revolution for robots and vehicles, particularly as it applies to sensors and networks.

Recent news

Recent publications

2020

Rotor-induced Airflow for Odor Source Detection on Nano-Quadcopters

Alexander Castro, Leo Peckerar, Timothy Horiuchi, Pamela Abshire

A nano-quadcopter system that performs odor detection in a windless indoor environment. The system uses the Crazyflie platform and incorporates a single commercially available metal oxide semiconductor gas sensor. The authors demonstrate that rotor-induced airflow enhances odor detection. It can be considered to be the quadrotor equivalent of sniffing behavior.

2020 IEEE SENSORS Conference

2020

Preparing for Geophysical Science Enabled by Crewed and Robotic Missions on the Surface of the Moon

Schmerr, N.; Richardson, J.; Ghent, R.; Siegler, M.; Young, K.; Wasser, M.; Whelley, P.; Buczkowski, D.; Carter, L.; Connor, C.; Connor, L.; Bleacher, J.; Fouch, M.; Baker, D.; Hurford, T.; Jozwiak, L.; Kruse, S.; Lekic, V.; Naids, A.; Porter, R. Montesi, L.; Richardson, D. C.; Rumpf, E.; Schorghofer, N.; Sunshine, J.; Goossens, S.; Whelley, N.; Wyrick, D.; Zhu, W.; Bell, E.; DeMartini, J.; Coan, D.; Akin, D.; Cohen, B.; Mazarico, E.; Neal, C.; Panning, M.; Petro, N.; Strauss, B.; Weber, R.; Glotch, T.; Hendrix, A.; Parker, A.; Wright, S.

Geophysics on the Moon will be an important tool for identifying key targets for geological prospecting, scientific sampling, ISRU, assessing hazards and risks to crews and infrastructure, and determining the deep workings of the lunar interior.

Lunar Surface Science Workshop, held virtually May 28-29, 2020

2023

UIVNAV: Underwater Information-driven Vision-based Navigation via Imitation Learning

Xiaomin Lin, Nare Karapetyan, Kaustubh Joshi, Tianchen Liu, Nikhil Chopra, Miao Yu, Pratap Tokekar, Yiannis Aloimonos

Autonomous navigation in the underwater environment is challenging due to limited visibility, dynamic changes, and the lack of a cost-efficient accurate localization system. We introduce UIVNAV, a novel end-to-end underwater navigation solution designed to drive robots over Objects of Interest (OOI) while avoiding obstacles, without relying on localization. UIVNAV uses imitation learning and is inspired by the navigation strategies used by human divers who do not rely on localization.

arXiv.org

Ajna: Generalized deep uncertainty for minimal perception on parsimonious robots

Nitin Sanket, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos

Robots are active agents that operate in dynamic scenarios with noisy sensors. Predictions based on these noisy sensor measurements often lead to errors and can be unreliable. To this end, roboticists have used fusion methods using multiple observations. Lately, neural networks have dominated the accuracy charts for percep- tion-driven predictions for robotic decision-making and often lack uncertainty metrics associated with the pre- dictions. Here, the authors present a mathematical formulation to obtain the heteroscedastic aleatoric uncertainty of any arbitrary distribution without prior knowledge about the data. The approach has no prior assumptions about the prediction labels and is agnostic to network architecture.

Science Robotics

Considerations for Minimizing Data Collection Biases for Eliciting Natural Behavior in Human-Robot Interaction

Snehesh Shrestha, Ge Gao, Cornelia Fermuller, Yiannis Aloimonos

About this paper, the authors write: "Many of us researchers take extra measures to control for known-unknowns. However, unknown-unknowns can, at best, be negligible, but otherwise, they could produce unreliable data that might have dire consequences in real-life downstream applications. Human-Robot Interaction standards informed by empirical data could save us time and effort and provide us with the path toward the robots of the future. To this end, we share some of our pilot studies, lessons learned, and how they affected the outcome of our experiments. While these aspects might not be publishable in themselves, we hope our work might save time and effort for other researchers towards their research and serve as additional considerations for discussion at the workshop."

github.io

NatSGD: A Dataset with Speech, Gestures, and Demonstrations for Robot Learning in Natural Human-Robot Interaction

Snehesh Shrestha, Yantian Zha, Ge Gao, Cornelia Fermuller, Yiannis Aloimonos

The work introduces NatSGD, a multimodal HRI dataset that contains human commands as speech and gestures, along with robot behavior in the form of synchronized demonstrated robot trajectories. These data enable HRI with Imitation Learning so that robots can learn to work with humans in challenging, real-life domains such as performing complex tasks in the kitchen.

github.io

2022

SeaDroneSim: Simulation of Aerial Images for Detection of Objects Above Water

Xiaomin Lin, Cheng Liu, Allen Pattillo, Miao Yu, Yiannis Aloimonos

SeaDroneSim is a new benchmark suite that can be used to create photo-realistic aerial image datasets with ground truth for segmentation masks of any given object.

Computer Vision Foundation 2023 WACV Workshop

Principles of Cognitive Vision

Yiannis Aloimonos, Giulio Sandini

Chapter 14 in the MIT Press book, Cognitive Robotics, edited by Angelo Cangelosi and Minoru Asada. This volume is part of the Intelligent Robotics and Autonomous Agents series.

In Cognitive Robotics (The MIT Press, 2022)

OysterNet: Enhanced Oyster Detection using Simulation

Xiaomin Lin, Nitin J. Sanket, Nare Karapetyan, Yiannis Aloimonos

A new way to mathematically model oysters and render images of oysters in simulation. This new method can boost detection performance with minimal real data, especially when used in conjunction with underwater robots.

arXiv.org

EVIMO2: An Event Camera Dataset for Motion Segmentation, Optical Flow, Structure from Motion, and Visual Inertial Odometry in Indoor Scenes with Monocular or Stereo Algorithms

Levi Burner, Anton Mitrokhin, Yiannis Aloimonos, Cornelia Fermüller

A new event camera dataset, EVIMO2, is introduced that improves on the popular EVIMO dataset by providing more data, from better cameras, in more complex scenarios.

arXiv.org

2021

EVPropNET: Detecting drone by finding propellers for mid-air landing and following

Yiannis Aloimonos, Cornelia Fermüller, Nitin Sanket, Chahat Deep Singh, Chethan Parameshwara, Guido C.H.E. de Croon

The authors use the most ubiquitous part of a drone—the propeller—as the basis for a new detection scheme that is 92% accurate.

2021 Robotics Science and Systems Conference (RSS 2021)

Detecting and Counting Oysters

Behzad Sadrfaridpour, Yiannis Aloimonos, Miao Yu, Yang Tao, Donald Webster

To test the idea that advancements in robotics and artificial intelligence offer the potential to improve the monitoring of oyster beds, the researchers prepared a remote operated underwater vehicle (ROV) with a camera and filmed in the Chesapeake Bay. They then used these videos to train convolutional neural networks (CNNs) to count oysters and track them in consecutive image frames so they are not identified multiple times.

arXiv.org

2020

MorphEyes: Variable Baseline Stereo for Quadrotor Navigation

Nitin J. Sanket, Chahat Deep Singh, Varun Asthana, Cornelia Fermüller, Yiannis Aloimonos

A framework for quadrotor navigation based on a stereo camera system whose baseline can be adapted on the fly.

arXiv.org

Unsupervised Learning of Dense Optical Flow, Depth and Egomotion with Event-Based Sensors

Chengxi Ye, Anton Mitrokhin, Cornelia Fermüller, James Yorke, Yiannis Aloimonos

Presents an unsupervised learning pipeline for dense depth, optical flow and egomotion estimation for autonomous driving applications, using the event-based output of the Dynamic Vision Sensor as input.

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Grasping in the Dark: Compliant Grasping using Shadow Dexterous Hand and BioTac Tactile Sensor

Kanishka Ganguly, Behzad Sadrfaridpour, Krishna Bhavithavya Kidambi, Cornelia Fermüller, Yiannis Aloimonos

An adaptive,compliant grasping strategy for robots using only tactile feedback.

arXiv.org

PRGFlow: Benchmarking SWAP-Aware Unified Deep Visual Inertial Odometry

Nitin J. Sanket, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos

The authors present a simple way to estimate ego-motion/odometry on an aerial robot using deep learning combining commonly found on-board sensors: a up/down-facing camera, an altimeter source and an IMU.

arXiv.org

EVDodgeNet: Deep Dynamic Obstacle Dodging with Event Cameras

Nitin J. Sanket, Chethan M. Parameshwara, Chahat Deep Singh, Ashwin V. Kuruttukulam, Cornelia Fermüller, Davide Scaramuzza, Yiannis Aloimonos

A deep learning-based solution for dodging multiple dynamic obstacles on a quadrotor with a single event camera and on-board computation.

2020 IEEE International Conference on Robotics and Automation (ICRA)

 

2023

Safe Collective Control under Noisy Inputs and Competing Constraints via Non-Smooth Barrier Functions

Clinton Enwerem, John Baras

Considers the problem of safely coordinating ensembles of identical autonomous agents to conduct complex missions with conflicting safety requirements and under noisy control inputs. Using non-smooth control barrier functions (CBFs) and stochastic model-predictive control as springboards, and by adopting an extrinsic approach where the ensemble is treated as a unified dynamic entity, the authors devise a method to synthesize safety-aware control inputs for uncertain collectives, drawing upon recent developments in Boolean CBF composition and extensions of CBFs to stochastic systems.

arXiv.org

RCMS: Risk-Aware Crash Mitigation System for Autonomous Vehicles

Faizan M. Tariq, David Isele, John Baras, Sangjae Bae

A risk-aware crash mitigation system (RCMS) to augment any existing motion planner (MP) that enables an autonomous vehicle to perform evasive maneuvers in high-risk situations and minimize the severity of collision if a crash is inevitable.

arXiv.org

Consensus-Based Leader-Follower Formation Tracking for Control-Affine Nonlinear Multiagent Systems

Clinton Enwerem, John Baras

Consensus-based control laws for multiagent formation tracking in finite-dimensional state space, with the agents represented by a more general class of dynamics: control-affine nonlinear systems. The agents also exchange information via a leader-follower communication topology modeled as an undirected and connected graph with a single leader node.

arXiv.org

Distributed Optimal Formation Control for an Uncertain Multiagent System in the Plane

Clinton Enwerem, John Baras, Danilo Romero

A distributed optimal multiagent control scheme for quadrotor formation tracking under localization errors. This control architecture is based on a leader-follower approach, where a single leader quadrotor tracks a desired trajectory while the followers maintain their relative positions in a triangular formation.

arXiv.org

Optimization-based Coordination and Control of Traffic Lights and Mixed Traffic in Multi-Intersection Environments

Nilesh Suriyarachchi, Rien Quirynen, John Baras, Stefano Di Cairano

Coordinating the flow of traffic through urban areas with multiple intersections is a complex problem whose solution has the potential to improve safety, increase throughput, and optimize energy efficiency. In addition to controlling traffic lights, the introduction of connected and automated vehicles (CAVs) offers opportunities in terms of additional sensing and actuation points within the traffic network.

This paper proposes a centralized and a decentralized implementation for the joint coordination and control of both traffic signals and mixed traffic, including CAVs and human-driven vehicles (HDVs), in a network of multiple connected traffic intersections. Mixed-integer linear programming (MILP) is used to compute safe control trajectories for both CAVs and traffic light signals, which minimize overall congestion and fuel consumption. Our approaches are validated using extensive traffic simulations on the SUMO platform and they are shown to provide improvements of around 32-60%, 90-96% and 40-60% in travel time, waiting time and fuel consumption, respectively, when compared to gap-based adaptive and timed traffic lights.

American Control Conference, 2023

Learning Agent Interactions from Density Evolution in 3D Regions with Obstacles

Amoolya Tirumalai, Christos Mavridis, John Baras

In this work, the researchers study the inverse problem of identifying complex flocking dynamics in a domain cluttered with obstacles. Their inspiration comes from animal flocks moving in complex ways with capabilities far beyond what current robots can do. Owing to the difficulty of observing and recovering the trajectories of the agents, the authors focus on the dynamics of their probability densities, which are governed by partial differential equations (PDEs), namely compressible Euler equations subject to non-local forces. The authors formulate the inverse problem of learning interactions as a PDE-constrained optimization problem of minimizing the squared Hellinger distance between the histogram of the flock and the distribution associated to their PDEs.

arXiv.org

Approximate Dynamic Programming for a Mean-Field Game of Traffic Flow: Existence and Uniqueness

Amoolya Tirumalai, John Baras

Highway vehicular traffic is an inherently multi-agent problem. Traffic jams can appear and disappear mysteriously. Here, the researchers develop a method for traffic flow control that is applied at the vehicular level via mean-field games.

IEEE 61st Conference on Decision and Control

Multi-agent Deep Reinforcement Learning for Shock Wave Detection and Dissipation using Vehicle-to-Vehicle Communication

Nilesh Suriyarachchi, Erfaun Noorani, Faizan Tariq, John Baras

Proposes the use of a multi-agent shared policy reinforcement learning algorithm which leverages communication between connected autonomous vehicles (CAVs) for improved observability of downstream traffic conditions. A key feature of this method is the ability to perform shock wave dissipation control without the need for global information and the applicability of this method to multi-lane mixed traffic highways of arbitrary structure.

IEEE 61st Conference on Decision and Control

SLAS: Speed and Lane Advisory System for Highway Navigation

Faizan Tariq, David Isele, John Baras, Sangjae Bae

This paper proposes a hierarchical autonomous vehicle navigation architecture, composed of a high-level speed and lane advisory system (SLAS) coupled with low-level trajectory generation and trajectory following modules. Specifically, the authors target a multi-lane highway driving scenario where an autonomous ego vehicle navigates in traffic. They propose a novel receding horizon mixed-integer optimization based method for SLAS with the objective to minimize travel time while accounting for passenger comfort.

IEEE 61st Conference on Decision and Control

2022

Model-Based Systems Engineering Simulation Framework for Robot Grasping

Praveen Kumar Menaka Sekar, John Baras

Constant rise in industrial usage of robots for commercial applications has led to the need for rapid, efficient, and reliable robotic system development processes. The idea of collaborative modeling and simulation provides different views to the stakeholders, enhancing the decision-making process towards design, development, and deployment. This paper applies Model-Based Systems Engineering (MBSE) principles to a robot grasping system performing a simple pick and place task; integrates Cameo Systems Modeler (CSM), CoppeliaSim robot simulator, and Gurobi Optimizer to facilitate and accelerate the system design process; and describes a simulation-based Verification & Validation (V&V) framework and performs design space exploration. This is an effort to integrate tools to reduce the burden on system design, integration, and deployment of dexterous grasping and manipulation systems.

INCOSE International Symposium Conference on Systems Engineering Research (CSER) 2022, Norwegian University of Science and Technology

GAMEOPT: Optimal Real-time Multi-Agent Planning and Control for Dynamic Intersections

Nilesh Suriyarachchi, Faizan Tariq, Christos Mavridis, John Baras

Intersections without traffic signals are the sites of 40% of all crashes, 50% of serious collisions, and 20% of fatalities. When motorists approach these intersections, which often have “four-way” stop signs, and feature “unprotected” left turns, they must make choices. If two vehicles approach at approximately the same time, or a driver decides not to stop or wait their turn, safety is at issue. In the future, autonomous vehicles equipped with vehicle-to-infrastructure (V2I) communication may be able to provide solutions for handling these scenarios. Such vehicles, called connected autonomous vehicles (CAVs), will be able to provide multiple sensing and actuation points in the traffic flow. GAMEOPT is a novel hybrid approach to cooperative intersection control for dynamic, multi-lane, unsignalized intersections.

arXiv.org

2021

Learning Swarm Interaction Dynamics from Density Evolution

Christos Mavridis, Amoolya Tirumalai, John Baras

Considers the problem of understanding the coordinated movements of biological or artificial swarms. The authors propose a learning scheme to estimate the coordination laws of interacting agents from observations of the swarm’s density over time. Dynamics of the swarm are described based on pairwise interactions according to a Cucker-Smale flocking model, and express the swarm’s density evolution as the solution to a system of mean-field hydrodynamic equations. A new family of parametric functions is proposed to model the pairwise interactions. The augmented system is incorporated in an iterative optimization scheme to learn the dynamics of the interacting agents from observations of the swarm’s density evolution over time.

arXiv.org

Weak Solutions to an Euler Alignment System in a Bounded Domain

Amoolya Tirumalai, Christos Mavridis, John Baras

Euler alignment systems appear as hydrodynamic limits of interacting self-propelled particle systems such as the (generalized) Cucker-Smale model. In this work, the authors study weak solutions to an Euler alignment system on smooth, bounded domains. The work falls into the general category of using insights gained from biological flocking and swarming to construct controls for large teams of UAVs.

arXiv.org

Real-Time Prioity-Based Cooperative Highway Merging for Heterogeneous Autonomous Traffic

Nilesh Suriyarachchi, Faizan Tariq, Christos Mavridis, John Baras

Highway on-ramp merge junctions remain a major bottleneck in transportation networks. However, with the introduction of Connected Autonomous Vehicles (CAVs) with advanced sensing and communication capabilities modern algorithms can capitalize on the cooperation between vehicles. This paper enhances highway merging efficiency by optimally coordinating CAVs in order to maximize the flow of vehicles while satisfying all safety constraints.

2021 IEEE International Intelligent Transportation Systems Conference (ITSC)

Model-Based Systems Engineering Applied to the Detection and Correction of Object Slippage within a Dexterous Robotic Hand

Charles Meehan, John Baras

The authors use Model-Based Systems Engineering techniques to verify system requirements and validate stakeholder requirements for the problem of detecting and correcting for object slippage within a dexterous five-fingered robotic hand.

2021 IEEE International Systems Conference (SysCon)

Order Effects of Measurements in Multi-Agent Hypothesis Testing

John Baras, Aneesh Raghavan

Revisits the structure of events in a multi-agent system and introduces necessary new models that incorporate incompatible events in formalism. These models are essential for building non-commutative probability models.

Communications in Information and Systems

2020

Detection of Dynamically Changing Leaders in Complex Swarms from Observed Dynamic Data

Christos Mavridis, Nilesh Suriyarachchi, John Baras

Considers the problem of defending against adversarial attacks from UAV swarms performing complex maneuvers,driven by multiple, dynamically changing, leaders.

GameSec 2020 Conference

Optimization-based Motion Planning and Runtime Monitoring for Robotic Agent with Space and Time Tolerances

Zhenyu Lin, John Baras

an optimization-based approach for robot planning, monitoring and self-correction problems under signal temporal logic specifications (STL).

21st IFAC World Congress

A Human-Robot Interface based on Surface Electroencephalographic Sensors

Christos Mavridis, John Baras, Kostas Kyriakopoulos

The authors propose a human-robot interface based on potentials recorded through surface Electroencephalographic sensors, aiming to decode human visual attention into motion in three-dimensional space.

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems

Joint Mobility-Aware UAV Placement and Routing in Multi-Hop UAV Relaying Systems

Anousheh Gholami, Nariman Torkzaban, John Baras, Chrysa Papagianni

The authors formulate the joint UAV placement and routing problem as a Mixed Integer Linear Program (MILP) and propose an approximation that leads to a LP rounding algorithm and achieves a balance between time-complexity and optimality.

arxiv.org

Metric Interval Temporal Logic based Reinforcement Learning with Runtime Monitoring and Self-Correction

Zhenyu Lin, John Baras

A modular Q-learning framework to deal with the robot task planning, runtime monitoring and self-correction problem.

2020 American Control Conference

2022

UIVNAV: Underwater Information-driven Vision-based Navigation via Imitation Learning

Xiaomin Lin, Nare Karapetyan, Kaustubh Joshi, Tianchen Liu, Nikhil Chopra, Miao Yu, Pratap Tokekar, Yiannis Aloimonos

Autonomous navigation in the underwater environment is challenging due to limited visibility, dynamic changes, and the lack of a cost-efficient accurate localization system. We introduce UIVNAV, a novel end-to-end underwater navigation solution designed to drive robots over Objects of Interest (OOI) while avoiding obstacles, without relying on localization. UIVNAV uses imitation learning and is inspired by the navigation strategies used by human divers who do not rely on localization.

arXiv.org

Cardynet: Deep Learning-based Navigation for Car-like Robots in Dynamic Environments

Gurtajbir Singh Herr, Lasitha Weerakoon, Miao Yu, Nikhil Chopra

The authors utilize an optimization based planner, namely TEB local planner, as the expert to generate high quality motion commands for a car-like robot operating in a simulated dynamic environment. They use these labels to train a deep neural network that learns to navigate. The deep learning-based planner is further augmented with safety margins to enhance its effectiveness in collision avoidance.

Proceedings of the ASME 2022 International Mechanical Engineering Congress and Exposition IMECE2022

Learning-based Autonomous Underwater Vehicle Navigation Following Human Actions in Confined Environment

Tianchen Liu, Miao Yu, Nikhil Chopra

Navigation of autonomous underwater vehicles (AUVs) is critical for many vital tasks operated in the aquatic environment. However, the navigation problem is challenging in confined environments like caves. Since humans can usually solve such a problem with minimal effort, the main objective of this paper is to implement learning-based neural network models for AUVs to learn from human actions in a confined environment. The Girona underwater dataset, acquired in a cave exploration experiment, was used for the learning task. The odometry and sonar data from the dataset were used to train two deep neural network models for AUV navigation through the underwater caves. The test data demonstrates that both models achieve relatively high accuracy for the navigation task.

IEEE OCEANS 2022 Conference, Hampton Roads, Va.

Cartographer glass: 2D Graph SLAM Framework using LiDAR for Glass Environments

Lasitha Weerakoon, Gurtajbir Singh Herr, Jasmine Blunt, Miao Yu, Nikhil Chopra

The authors study algorithms for detecting and including glass objects in an optimization-based Simultaneous Localization and Mapping (SLAM) algorithm for safe and reliable robot navigation. They utilize a simple and computationally inexpensive glass detection scheme for detecting glass objects and present the methodology to incorporate the identified objects into the occupancy grid maintained by such an algorithm (Google Cartographer).

arXiv.org

Passivity-Based Task Space Control of Hybrid Rigid-Soft (HyRiSo) Robots with Parametric Uncertainty

Lasitha Weerakoon, Nikhil Chopra

This paper introduces a novel robotic system, coined as a hybrid rigid-soft (HyRiSo) robot composed of rigid links and soft links serially attached. HyRiSo combines the benefits of the dexterity of a soft manipulator with the support capability of a classical stiff arm.

researchgate.net

Information System for Detecting Strawberry Fruit Locations and Ripeness Conditions in a Farm

Tianchen Liu, Nikhil Chopra, Jayesh Samtani

Many strawberry growers in some areas of the United States rely on customers to pick the fruits during the peak harvest months. Unfavorable weather conditions such as high humidity and excessive rainfall can quickly promote fruit rot and diseases. This study establishes an elementary farm information system to demonstrate timely information on the farm and fruit conditions (ripe, unripe) to the growers. The information system processes a video clip or a sequence of images from a camera to provide a map which can be viewed to estimate quantities of strawberries at different stages of ripeness. The farm map is built by state-of-the-art vision-based simultaneous localization and mapping (SLAM) techniques, which can generate the map and track the motion trajectory using image features. It can help farm labor direct traffic to specific strawberry locations within a farm where fruits need to be picked, or where berries need to be removed. The obtained system can help reduce farm revenue loss and promote sustainable crop production.

Proceedings of the 2022 Biology and Life Sciences Forum

Robotic Parasitic Array Control for Increased RSS in Non-Line-of-Sight

Jeffrey Twigg, Fikadu Dagefu, Nikhil Chopra, Brian Sadler

Safety, security, and rescue missions typically occur in environments where unreliable wireless communication impedes cooperative human and robot missions. A low-VHF band antenna array formed through the coordination of ground robots has the potential to transmit and receive signals more reliably at longer ranges in these environments. The authors advance this idea by developing methods for controlling a robot forming a low-VHF antenna array to improve or maintain wireless communication with other agents in these complex environments. They propose and test methods for orienting this robotic antenna array to increase received signal strength (RSS) in a complex environment and show that creating and actuating this array improves RSS in line-of-sight and non-line-of-sight conditions.

IEEE Robotics and Automation Letters

Co-Design of Lipschitz Nonlinear Systems

Prasad Vilas Chanekar, Nikhil Chopra

Empirical experiences have shown that simultaneous (rather than conventional sequential) plant and controller design procedure leads to an improvement in performance and saving of plant resources. Such a simultaneous synthesis procedure is called a “co-design.” This paper studies the co-design problem for a class of Lipschitz nonlinear dynamical systems having a quadratic control objective and state-feedback controller. It proposes time-independent reformulation of a co-design optimization problem whose constraints ensure stability of a system. The authors also present a gradient-based co-design solution procedure which involves system coordinate transformation and whose output is a provably stable solution for the original system. They show the efficacy of the solution procedure through co-design of a single-link robot.

arXiv.org

2021

Swing up Control of a Soft Inverted Pendulum with Revolute Base

Lasitha Weerakoon, Nikhil Chopra

The paper introduces a soft robotic system that is a soft inverted pendulum with a revolute joint at the base. The revolute joint is not actuated. The soft body is hypothesized to be of constant curvature and it is actuated. Motivated by swing up controllers for classical underactuated systems, a switching-based swing up and stabilization control of the proposed soft robot system is studied. The authors demonstrate that the swing up control guides the soft robot to the desired energy level, which is the upright position.

researchgate.net

2020

Accelerating Distributed SGD for Linear Regression using Iterative Pre-Conditioning

Kushal Chakrabarti, Nirupam Gupta, Nikhil Chopra

This paper considers the multi-agent distributed linear least-squares problem. The authors extend the idea of iterative pre-conditioning to stochastic settings, where the server updates the estimate and the iterative pre-conditioning matrix-based on a single randomly selected data point at every iteration.

arXiv.org

Communication Maintenance of Robotic Parasitic Antenna Arrays

Jeffrey Twigg, Nikhil Chopra, Brian Sadler

Recent developments in low-VHF antenna design and parasitic antenna array research show it is possible to form multi-robot antenna arrays that can extend communication range in urban and indoor environments. This paper proposes a generalized version of a disk model that describes a parasitic array. This model is then integrated into a Fiedler maximization approach for maintaining communication.

IEEE Robotics and Automation Letters

Bilateral Teleoperation of Soft Robots under Piecewise Constant Curvature Hypothesis: An Experimental Investigation

Lasitha Weerakoon, Nikhil Chopra

This paper investigates an adaptive task space bilateral teleoperation framework for soft robots with dynamic uncertainties. It assumes a non-redundant rigid master manipulator and a redundant soft slave manipulator under the piecewise constant curvature hypothesis.

2020 American Control Conference

2019

Robotic Parasitic Array Optimization in Outdoor Environments

Jeffrey Twigg, Fikadu Dagefu, Nikhil Chopra, Brian M. Sadler

The paper describes a two-element parasitic array formed by two ground robots and proposes a technique by which this array can optimize its gain in a direction of interest online.

2019 IEEE International Symposium on Safety, Security, and Rescue Robotics

2023

On the Equivalence of Multi-Agent 2D Coverage Control and Leader-Follower Consensus Network

Xiaotian Xu, Guangyao Shi, Pratap Tokekar, Yancy Diaz-Mercado

A distributed interactive framework to provide high-level position instructions for multi-robot aerial cinematography based on coverage over a hemisphere. The control strategy is based on optimization of the coverage functional and geometric relationships over a hemisphere. It enables multiple Unmanned Aerial Vehicles (UAVs) to coordinate their motion while tracking a dynamic (real or virtual) target, and can accommodate high-level human inputs to influence UAV concentration.

2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

2022

Contraction Analysis of Multi-Agent Control for Guaranteed Capture of a Faster Evader

Phillip Rivera-Ortiz, Andrew Frommer, Yancy Diaz-Mercado

A verifiable condition for the selection of a sufficient number of pursuers to capture a faster evader. The condition is based on the tracking performance of a multi-agent control scheme.

2022 IEEE 61st Conference on Decision and Control (CDC)

Interactive Multi-Robot Aerial Cinematography Through Hemispherical Manifold Coverage

Phillip Rivera, Marin Kobilarov, Yancy Diaz-Mercado

Scalable pursuer coordination for reach-avoid games against a fast evader are developed leveraging coverage control over manifolds. The maintenance of a manifold, termed defense surface, prevents the evader and its target from occupying the same half-space and shown sufficient as a cooperative capture strategy. Nonlinear control synthesis continually reconfigures the pursuers to enable a defense surface via coverage. Simulation results empirically validate that the proposed condition serves as a surrogate objective for pursuer team coordination.

IEEE Transactions on Automatic Control

Pursuer Coordination Against a Fast Evader via Coverage Control

Phillip Rivera, Marin Kobilarov, Yancy Diaz-Mercado

Scalable pursuer coordination for reach-avoid games against a fast evader are developed leveraging coverage control over manifolds. The maintenance of a manifold, termed defense surface, prevents the evader and its target from occupying the same half-space and shown sufficient as a cooperative capture strategy. Nonlinear control synthesis continually reconfigures the pursuers to enable a defense surface via coverage. Simulation results empirically validate that the proposed condition serves as a surrogate objective for pursuer team coordination.

IEEE Transactions on Automatic Control

Control of Magnetic Surgical Robots With Model-Based Simulators and Reinforcement Learning

Yotam Barnoy, Onder Erin, Suraj Raval, Will Pryor, Lamar O. Mair, Irving N. Weinberg, Yancy Diaz-Mercado, Axel Krieger, Yancy Diaz-Mercado, Gregory Hager

Magnetically manipulated medical robots are a promising alternative to current robotic platforms, allowing for miniaturization and tetherless actuation. Controlling such systems autonomously may enable safe, accurate operation. However, classical control methods require rigorous models of magnetic fields, robot dynamics, and robot environments, which can be difficult to generate. Model-free reinforcement learning (RL) offers an alternative that can bypass these requirements. Here, the authors apply RL to a robotic magnetic needle manipulation system.

IEEE Transactions on Medical Robotics and Bionics

Self-Triggered Coverage Control for Mobile Sensors

Erick J. Rodríguez-Seda, Xiaotian Xu, Josep M. Olm, Arnau Dòria-Cerezo, Yancy Diaz-Mercado

A distributed asynchronous self-triggered control policy for centroidal Voronoi coverage control that is shown to decrease the sampling or communication instants among agents without degrading the performance of the mobile sensor network.

IEEE Transactions on Robotics

Enhanced Accuracy in Magnetic Actuation: Closed-Loop Control of a Magnetic Agent With Low-Error Numerical Magnetic Model Estimation

Onder Erin, Suraj Raval, Trevor J. Schwehr, Will Pryor, Yotam Barnoy, Adrian Bell, Xiaolong Liu, Lamar Mair, Irving Weinberg, Axel Krieger, Yancy Diaz-Mercado

Magnetic actuation holds promise for wirelessly controlling small, magnetic surgical tools and may enable the next generation of ultra minimally invasive surgical robotic systems. Precise torque and force exertion are required for safe surgical operations and accurate state control. Dipole field estimation models perform well far from electromagnets but yield large errors near coils. Thus, manipulations near coils suffer from severe (10x) field modeling errors. The authors experimentally quantify closed-loop magnetic agent control performance by using both a highly erroneous dipole model and a more accurate numerical magnetic model to estimate magnetic forces and torques for any given robot pose in 2D. Such improved magnetic modelling is crucial for systems requiring robust estimates of magnetic forces for positioning agents, particularly in force-sensitive environments like surgical manipulation.

2022 IEEE Robotics and Automation Letters

Effectiveness of Augmented Reality for Human Swarm Interactions

Sarjana Oradiambalam Sachidanandam, Sara Honarvar, Yancy Diaz-Mercado

Human-Swarm Interaction (HSI) is a fast-growing research area in swarm robotics. The researchers investigate the effectiveness of augmented reality-based interfaces as tools for remote interaction in HSI systems. They develop a simple AR-based interface and evaluate its effectiveness against an unaugmented interface, using remote human user studies.

2022 International Conference on Robotics and Automation (ICRA)

2021

Spoken Language Interaction for Robotics: Research Issues and Recommendations

M. Marge, C. Espy-Wilson, N. Ward, A. Alwan, Y. Artzi, M. Bansal, G. Blankenship, J. Chai, H. Daumé III, D. Dey , M. Harper, T.Howard, C. Kennington I. Kruijff-Korbayová, D. Manocha, C. Matuszek, R. Mead, R.Mooney, R. K. Moore, M. Ostendorf, H. Pon-Barry, A. Rudnicky, M. Scheutz, R. St. Amant, T. Sun, S. Tellex, D. Traum, Z. Yu

With robotics rapidly advancing, more effective human–robot interaction is increasingly needed to realize the full potential of robots for society. While spoken language must be part of the solution, our ability to provide spoken language interaction capabilities is still very limited. In this article, based on the report of an interdisciplinary workshop convened by the National Science Foundation, the authors identify key scientific and engineering advances needed to enable effective spoken language interaction with robotics. The authors make 25 recommendations, involving eight general themes: putting human needs first, better modeling the social and interactive aspects of language, improving robustness, creating new methods for rapid adaptation, better integrating speech and language with other communication modalities, giving speech and language components access to rich representations of the robot’s current knowledge and state, making all components operate in real time, and improving research infrastructure and resources. Research and development that prioritizes these topics will, we believe, provide a solid foundation for the creation of speech-capable robots that are easy and effective for humans to work with.

Computer Speech & Language, January 2022

2023

Ajna: Generalized deep uncertainty for minimal perception on parsimonious robots

Nitin Sanket, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos

Robots are active agents that operate in dynamic scenarios with noisy sensors. Predictions based on these noisy sensor measurements often lead to errors and can be unreliable. To this end, roboticists have used fusion methods using multiple observations. Lately, neural networks have dominated the accuracy charts for percep- tion-driven predictions for robotic decision-making and often lack uncertainty metrics associated with the pre- dictions. Here, the authors present a mathematical formulation to obtain the heteroscedastic aleatoric uncertainty of any arbitrary distribution without prior knowledge about the data. The approach has no prior assumptions about the prediction labels and is agnostic to network architecture.

Science Robotics

Considerations for Minimizing Data Collection Biases for Eliciting Natural Behavior in Human-Robot Interaction

Snehesh Shrestha, Ge Gao, Cornelia Fermuller, Yiannis Aloimonos

About this paper, the authors write: "Many of us researchers take extra measures to control for known-unknowns. However, unknown-unknowns can, at best, be negligible, but otherwise, they could produce unreliable data that might have dire consequences in real-life downstream applications. Human-Robot Interaction standards informed by empirical data could save us time and effort and provide us with the path toward the robots of the future. To this end, we share some of our pilot studies, lessons learned, and how they affected the outcome of our experiments. While these aspects might not be publishable in themselves, we hope our work might save time and effort for other researchers towards their research and serve as additional considerations for discussion at the workshop."

github.io

NatSGD: A Dataset with Speech, Gestures, and Demonstrations for Robot Learning in Natural Human-Robot Interaction

Snehesh Shrestha, Yantian Zha, Ge Gao, Cornelia Fermuller, Yiannis Aloimonos

The work introduces NatSGD, a multimodal HRI dataset that contains human commands as speech and gestures, along with robot behavior in the form of synchronized demonstrated robot trajectories. These data enable HRI with Imitation Learning so that robots can learn to work with humans in challenging, real-life domains such as performing complex tasks in the kitchen.

github.io

2021

EVPropNET: Detecting drone by finding propellers for mid-air landing and following

Yiannis Aloimonos, Cornelia Fermüller, Nitin Sanket, Chahat Deep Singh, Chethan Parameshwara, Guido C.H.E. de Croon

The authors use the most ubiquitous part of a drone—the propeller—as the basis for a new detection scheme that is 92% accurate.

2021 Robotics Science and Systems Conference (RSS 2021)

2020

MorphEyes: Variable Baseline Stereo for Quadrotor Navigation

Nitin J. Sanket, Chahat Deep Singh, Varun Asthana, Cornelia Fermüller, Yiannis Aloimonos

A framework for quadrotor navigation based on a stereo camera system whose baseline can be adapted on the fly.

arXiv.org

Unsupervised Learning of Dense Optical Flow, Depth and Egomotion with Event-Based Sensors

Chengxi Ye, Anton Mitrokhin, Cornelia Fermüller, James Yorke, Yiannis Aloimonos

Presents an unsupervised learning pipeline for dense depth, optical flow and egomotion estimation for autonomous driving applications, using the event-based output of the Dynamic Vision Sensor as input.

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Grasping in the Dark: Compliant Grasping using Shadow Dexterous Hand and BioTac Tactile Sensor

Kanishka Ganguly, Behzad Sadrfaridpour, Krishna Bhavithavya Kidambi, Cornelia Fermüller, Yiannis Aloimonos

An adaptive,compliant grasping strategy for robots using only tactile feedback.

arXiv.org

PRGFlow: Benchmarking SWAP-Aware Unified Deep Visual Inertial Odometry

Nitin J. Sanket, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos

The authors present a simple way to estimate ego-motion/odometry on an aerial robot using deep learning combining commonly found on-board sensors: a up/down-facing camera, an altimeter source and an IMU.

arXiv.org

EVDodgeNet: Deep Dynamic Obstacle Dodging with Event Cameras

Nitin J. Sanket, Chethan M. Parameshwara, Chahat Deep Singh, Ashwin V. Kuruttukulam, Cornelia Fermüller, Davide Scaramuzza, Yiannis Aloimonos

A deep learning-based solution for dodging multiple dynamic obstacles on a quadrotor with a single event camera and on-board computation.

2020 IEEE International Conference on Robotics and Automation (ICRA)

2021

Bayesian Optimization for Design of Multi-Actuator Soft Catheter Robots

Seyede Fatemeh Ghoreishi, Ryan Sochol, Dheeraj Gandi, Axel Krieger, Mark Fuge

The authors propose modeling multi-actuator soft catheters to enable alignment with desired vessel shapes near the target area. They develop mathematical models to simulate the catheter’s positioning due to the moments exerted by multiple pneumatic actuators along the catheter and use those models to compare optimization approaches that can achieve catheter alignment along a desired vessel shape.

IEEE Transactions on Medical Robotics and Bionics

2023

Cognitive-motor performance assessment during in-person and remote practice of action sequences

Alexandra Shaver, Calvin Lu, Christopher Gaskins, James Reggia, James Purtilo, Rodolphe J. Gentili

The development and use of virtual systems to learn and perform remotely complex action sequences has many applications from rehabilitation to military training. The examination of human cognitive-motor control and learning processes in individuals executing a task physically in-person and remotely via a virtual system can help with the design of such systems as well as inform human performance. An important application of such virtual systems is the remote assessment of cognitive-motor performance and learning for the neurorehabilitation of individuals with neural deficits (e.g., veterans). This study is a first step that examined performance, mental workload, and fatigue in healthy individuals during remote practice of action sequences compared to physical in-person training.

2023 11th International IEEE/EMBS Conference on Neural Engineering (NER)

2022

NeuroCERIL: Robotic Imitation Learning via Hierarchical Cause-Effect Reasoning in Programmable Attractor Neural Networks

Gregory Davis, Garrett Katz, James Reggia, Rodolphe Gentili

The researchers explore the viability of developing purely-neural controllers for social robots that learn to imitate by reasoning about the underlying intentions of demonstrated behaviors. NeuroCERIL is a brain-inspired neurocognitive architecture that uses a novel hypothetico-deductive reasoning procedure to produce generalizable and human-readable explanations for demonstrated behavior.

arXiv.org

2023

Using Metareasoning on a Mobile Ground Robot to Recover from Path Planning Failures

Sidney Molnar, Matt Mueller, Robert Macpherson, Lawrence Rhoads, Jeffrey Herrmann

This paper describes metareasoning approaches that enable a robot to select a new path planning algorithm when the current planning algorithm cannot find a feasible solution.

UMD DRUM Library

Distributed Task Allocation Algorithms for Multi-Agent Systems With Very Low Communication

Akshay Bapat, Bharath Reddy Bora, Jeffrey Herrmann, Shapour Azarm, Huan Xu, Michael Otte

Explores the problem of task allocation among networked multi-robot systems when communication conditions are poor. The authors develop solutions that ensure all tasks are eventually completed—even if some agents in the network are destroyed. Their two new task allocation algorithms assume communication may not happen, but benefit the robotic agents and their missions whenever communication is successful.

IEEE Access

2020

Decentralized Task Allocation in Multi-Agent Systems Using a Decentralized Genetic Algorithm

Ruchir Patel, Eliot Rudnick-Cohen, Shapour Azarm, Michael Otte, Huan Xu, Jeffrey W. Herrmann

In multi-agent collaborative search missions, task allocation is required to determine which agents will perform which tasks. The paper proposes a new approach for decentralized task allocation based on a decentralized genetic algorithm.

2020 International Conference on Robotics and Automation

Experimental Comparison of Decentralized Task Allocation Algorithms Under Imperfect Communication

Sharan Nayak, Mohamed Khalid M. Jaffar, Estefany Carrillo, Suyash Yeotikar, Eliot Rudnick-Cohen, Ruchir Patel, Jeffrey Herrmann, Huan Xu, Shapour Azarm, Michael Otte

An experimental comparison of the performance of five state-of-the-art decentralized task allocation algorithms under imperfect communication conditions for teams of unmanned aerial vehicles (UAVs).

IEEE Robotics and Automation Letters

Data-driven Metareasoning for Collaborative Autonomous Systems

Jeffrey Herrmann

A novel data-driven metareasoning approach that generates a metareasoning policy that agents in a multi-agent system can use whenever they must collaborate to assign tasks.

 

 

 

 

 

 

 

 

ISR Technical Report, DRUM

2020

Rotor-induced Airflow for Odor Source Detection on Nano-Quadcopters

Alexander Castro, Leo Peckerar, Timothy Horiuchi, Pamela Abshire

A nano-quadcopter system that performs odor detection in a windless indoor environment. The system uses the Crazyflie platform and incorporates a single commercially available metal oxide semiconductor gas sensor. The authors demonstrate that rotor-induced airflow enhances odor detection. It can be considered to be the quadrotor equivalent of sniffing behavior.

2020 IEEE SENSORS Conference

2023

Flocks, Games, and Cognition: A Geometric Approach

Udit Halder, Vidya Raju, Matteo Mischiati, Biswadip Dey, P. S. Krishnaprasad

Avian flocks display a wide variety of flight behaviors, including steady directed translation of center of mass, rapid change of overall morphology, re-shuffling of positions of individuals within a persistent form, etc. These behaviors may be viewed as flock-scale strategies, emerging from interactions between individuals, accomplishing some collective adaptive purpose such as finding a roost, or mitigating the danger from predator attacks. While we do not conceive the flock as a single cognitive agent, the moment-to-moment decisions of individuals, influenced by their neighbors, appear as if to realize collective strategies that are cognizant of purpose. In this paper, we identify the actions of the flock as allocation of energetic resources, and thereby associate a cognitive cost to behavior. Our notion of cognitive cost is a measure of the temporal variability of such resource allocation. Using a recently developed natural geometric approach to kinetic energy allocation, we map the flock behavior to a temporal signature on the standard (probability) simplex. From the signature of a flocking event, we calculate the cognitive cost as a solution to an optimal control problem based on a game-theoretic model. Alternatively, one can associate to a signature an entropic cost. These two-cost measures, when applied to data on starling flocks, show a consistent spread in value across events, and we suggest the possibility that higher cost may arise from flock response to predator attacks.

Systems & Control Letters

AdVENTR: Autonomous Robot Navigation in Complex Outdoor Environments

Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Mohamed Elnoor, Dinesh Manocha

AdVENTR is a new system for autonomous robot navigation in unstructured outdoor environments with uneven and vegetated terrains. The general approach can enable both wheeled and legged robots to handle outdoor terrain complexity including unevenness, surface properties like poor traction, granularity, obstacle stiffness, etc. The system uses data from sensors including RGB cameras, 3D Lidar, IMU, robot odometry, and pose information with efficient learning-based perception and planning algorithms that can execute on edge computing hardware. AdVENTR uses a scene-aware switching method to perceive the environment for navigation at any time instant and dynamically switches between multiple perception algorithms.

arXiv.org

LANCAR: Leveraging Language for Context-Aware Robot Locomotion in Unstructured Environments

Chak Lam Shek, Xiyang Wu, Dinesh Manocha, Pratap Tokekar, and Amrit Singh Bedi

Optimal locomotion policy can be context-dependent by using the contextual information of encountered terrains in decision-making. Humans can interpret the environmental context for robots, but the ambiguity of human language makes it challenging to use in robot locomotion directly. In this paper, the authors propose a novel approach, LANCAR, that introduces a context translator that works with reinforcement learning (RL) agents for context-aware locomotion. The formulation allows a robot to interpret the contextual information from environments generated by human observers or Vision-Language Models (VLM) with Large Language Models (LLM) and use this information to generate contextual embeddings.

arXiv.org

VAPOR: Legged Robot Navigation in Unstructured Outdoor Environments using Offline Reinforcement Learning

Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Mohamed Elnoor, Dinesh Manocha

VAPOR is a new method for autonomous legged robot navigation in unstructured, densely vegetated outdoor environments using offline Reinforcement Learning (RL).

arXiv.org

MTG: Mapless Trajectory Generator with Traversability Coverage for Outdoor Navigation

Jing Liang, Peng Gao, Xuesu Xiao, Adarsh Jagan Sathyamoorthy,Mohamed Elnoor, Ming Lin, Dinesh Manocha

A learning algorithm for trajectory generation for outdoor robot navigation. The goal is to compute collision-free paths that also satisfy environment-specific traversability constraints.

arXiv.org

Using Lidar Intensity for Robot Navigation

Adarsh Jagan Sathyamoorthy, Kasun Weerakoon, Mohamed Elnoor, Dinesh Manocha

The authors present Multi-Layer Intensity Map, a novel 3D object representation for robot perception and autonomous navigation. It consists of multiple stacked layers of 2D grid maps each derived from reflected point cloud intensities corresponding to a certain height interval. The different layers of the intensity maps can be used to simultaneously estimate obstacles’ height, solidity/density, and opacity. The work demonstrates that they can help accurately differentiate obstacles that are safe to navigate through from ones that must be avoided in indoor and outdoor environments.

arXiv.org

CGLR: Dense Multi-Agent Navigation Using Voronoi Cells and Congestion Metric-based Replanning

Senthil Hariharan Arul, Dinesh Manocha

CGLR is a decentralized path-planning algorithm for navigating multiple differential-drive robots in dense environments. In contrast to prior decentralized methods, here the authors propose a novel congestion metric-based replanning that couples local and global planning techniques to efficiently navigate in scenarios with multiple corridors.

2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

ProNav: Proprioceptive Traversability Estimation for Autonomous Legged Robot Navigation in Outdoor Environments

Mohamed Elnoor, Adarsh Jagan Sathyamoorthy, Kasun Weerakoon, Dinesh Manocha

ProNav uses proprioceptive signals for traversability estimation in challenging outdoor terrains for autonomous legged robot navigation.

arXiv.org

Human Trajectory Forecasting with Explainable Behavioral Uncertainty

Jiangbei Yue, Dinesh Manocha, He Wang

Human trajectory forecasting helps to understand and predict human behaviors, enabling applications from social robots to self-driving cars, and therefore has been heavily investigated. Most existing methods can be divided into model-free and model-based methods. Model-free methods offer superior prediction accuracy but lack explainability, while model-based methods provide explainability but cannot predict well. Combining both methodologies, the authors propose a new Bayesian Neural Stochastic Differential Equation model BNSP-SFM, where a behavior SDE model is combined with Bayesian neural networks (BNNs).

arXiv.org

Dealing with Sparse Rewards in Continuous Control Robotics via Heavy-Tailed Policy Optimization

Souradip Chakraborty, Amrit Singh Bedi, Kasun Weerakoon, Prithvi Poddar, Alec Koppel, Pratap Tokekar, Dinesh Manocha

A novel Heavy-Tailed Stochastic Policy Gradient (HT-PSG) algorithm to deal with the challenges of sparse rewards in continuous control problems.

2023 IEEE International Conference on Robotics and Automation (ICRA)

DifFAR: Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition

Divya Kothandaraman, Ming Lin, Dinesh Manocha

DifFAR is a learning algorithm for human activity recognition in videos. The approach is designed for UAV videos, which are mainly acquired from obliquely placed dynamic cameras that contain a human actor along with background motion. Typically, the human actors occupy less than one-tenth of the spatial resolution. DifFAR simultaneously harnesses the benefits of frequency domain representations, a classical analysis tool in signal processing, and data driven neural networks. The authors build a differentiable static-dynamic frequency mask prior to model the salient static and dynamic pixels in the video, crucial for the underlying task of action recognition.

2023 IEEE International Conference on Robotics and Automation (ICRA)

Terrain-aware autonomous robot navigation in outdoor environments

Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Dinesh Manocha

A method to perceive a terrain’s geometric and other surface properties for efficient motion planning in outdoor environments. The method incorporates two perception branches to identify the terrain’s elevation and roughness separately. The first branch uses an elevation map created using LiDAR point clouds to compute a cost map that represents critical elevation changes. The second branch uses a vision-based cost map trained using RGB images, IMU, and robot odometry. Then, least-cost waypoints are calculated on a combined cost map and are followed using the Dynamic Window Approach (DWA). This planner navigates along the least-cost waypoints while adaptively varying the velocities to reduce vibration. The authors evaluate the method’s performance on a Husky robot in real-world environments and observe that the method leads to higher success rates, and lower vibrations compared to state-of-the-art methods.

SPIE Proceedings of the Open Architecture/Open Business Model Net-Centric Systems and Defense Transformation 2023

Frequency-based aerial video recognition

Divya Kothandaraman, Xijun Wang, Tianrui Guan, Sean Hu, Ming Lin, Dinesh Manocha

Aerial video recognition is challenging due to various factors. Prior work on action recognition imposes constraints in terms of unavailability of object detection bounding box ground-truth inhibiting the application of localization models and computational constraints preventing the usage of expensive space-time self-attention. Optical flow and pretrained models for detecting human actor performing action do not work too well due to domain gap issues. Here, the authors present a frequency-domain space-time attention method that encapsulates long-range space-time dependencies by emulating the weighted outer product in the frequency domain. They also show a frequency-based object background disentanglement method to inherently separate out the moving human actor from the background and a mathematical model for static salient regions and an identity loss function to learn disentangled features in a differentiable manner.

SPIE Proceedings of the Open Architecture/Open Business Model Net-Centric Systems and Defense Transformation 2023

PMI Sampler: Patch similarity guided frame selection for Aerial Action Recognition

Ruiqi Xian, Xijun Wang, Divya Kothandarama, Dinesh Manocha

A new algorithm for selection of informative frames in video action recognition. The approach is designed for aerial videos captured using a moving camera where human actors occupy a small spatial resolution of video frames. The algorithm utilizes the motion bias within aerial videos, which enables the selection of motion-salient frames. The authors introduce the concept of patch mutual information (PMI) score to quantify the motion bias between adjacent frames, by measuring the similarity of patches.

arXiv.org

3D-Online Generalized Sensed Shape Expansion: A Probabilistically Complete Motion Planner in Obstacle-Cluttered Unknown Environments

Vrushabh Zinage, Senthil Hariharan Arul, Dinesh Manocha, Satadal Ghosh

An online motion planning algorithm (3D-OGSSE) for generating smooth, collision-free trajectories over multiple planning iterations for a 3-D agent operating in an unknown, obstacle-cluttered, 3-D environment. In each planning iteration, 3D-OGSSE constructs an obstacle-free region termed “generalized sensed shape” based on the locally-sensed environment information and the notion of generalized shape. A collision-free path is computed by sampling points in the generalized sensed shape and is used to generate a smooth, time-parametrized trajectory by minimizing snap. The generated trajectory at every planning iteration is constrained to lie within generalized sensed shape, which ensures the agent maneuvers in locally obstacle-free space. As the agent reaches the boundary of the generalized sensed shape in a planning iteration, a re-plan is triggered by a receding horizon planning mechanism that also enables the initialization of the next planning iteration.

IEEE Robotics and Automation Letters

CROSSLOC3D: Aerial-Ground Cross-Source 3D Place Recognition

Tianrui Guan, Aswath Muthuselvam, Montana Hoover, Xijun Wang, Jing Liang, Adarsh Jagan Sathyamoorthy, Damon Conover, Dinesh Manocha

This is a novel 3D place recognition method that solves a large-scale point matching problem in a cross-source setting. Cross-source point cloud data corresponds to point sets captured by depth sensors with different accuracies or from different distances and perspectives. We address the challenges in terms of developing 3D place recognition methods that account for the representation gap between points captured by different sources. Our method handles cross-source data by utilizing multi-grained features and selecting convolution kernel sizes that correspond to most prominent features.

arXiv.org

VERN: Vegetation-aware Robot Navigation in Dense Unstructured Outdoor Environments

Adarsh Jagan Sathyamoorthy, Kasun Weerakoon, Tianrui Guan, Mason Russell, Damon Conover, Jason Pusey, Dinesh Manocha

This is a method for autonomous legged robot navigation in densely vegetated environments with a variety of pliable/traversable and non-pliable/untraversable vegetation. The authors present a few-shot learning classifier that can be trained on a few hundred RGB images to differentiate flora that can be navigated through, from the ones that must be circumvented. Using the vegetation classification and 2D lidar scans, this method constructs a vegetation-aware traversability cost map that accurately represents the pliable and non-pliable obstacles with lower, and higher traversability costs, respectively. The cost map construction accounts for misclassifications of the vegetation and further lowers the risk of collisions, freezing and entrapment in vegetation during navigation.

arXiv.org

Real-Time Decentralized Navigation of Nonholonomic Agents Using Shifted Yielding Area

Liang He, Zherong Pan, Dinesh Manocha

A lightweight, decentralized algorithm for navigating multiple nonholonomic agents through challenging environments with narrow passages. The key idea is to allow agents to yield to each other in large open areas instead of narrow passages, to increase the success rate of conventional decentralized algorithms.

arXiv.org

RE-MOVE: An Adaptive Policy Design Approach for Dynamic Environments via Language-Based Feedback

Souradip Chakraborty, Kasun Weerakoon Prithvi Poddar, Pratap Tokekar, Amrit Singh Bedi, Dinesh Manocha

Reinforcement learning-based policies for continuous control robotic navigation tasks often fail to adapt to changes in the environment during real-time deployment, which may result in catastrophic failures. To address this limitation, the authors propose a novel approach called RE-MOVE (REquest help and MOVE on), which uses language-based feedback to adjust trained policies to real-time changes in the environment.

arXiv.org

Dynamic EM Ray Tracing for Large Urban Scenes with Multiple Receivers

Ruichen Wang, Dinesh Manocha

A novel ray tracing-based radio propagation algorithm that can handle large urban scenes with hundreds or thousands of dynamic objects and receivers.

arXiv.org

DS-MPEPC: Safe and Deadlock-Avoiding Robot Navigation in Cluttered Dynamic Scenes

Senthil Hariharan Arul, Jong Jin Park, Dinesh Manocha

An algorithm for safe robot navigation in complex dynamic environments using a variant of model predictive equilibrium point control.

arXiv.org

An efficient acyclic contact planner for multiped robots

Steve Tonneau, Andrea Del Prete, Julien Pettre, Chonhyon Park, Dinesh Manocha, Nicolas Mansard

A framework capable of producing contact plans describing complex multiped motions (including humanoid): standing up, climbing stairs using a handrail, crossing rubble and getting out of a car. The framework answers a need demonstrated at the DARPA Robotics Challenge, where the lack of an automatic acyclic contact planner was recognized as a major issue. Here the novel key idea is the reachability condition.

HAL archive

Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Based Zero-Shot Object Navigation

Vishnu Sashank Dorbala, James Mullen, Dinesh Manocha

LGX is a novel algorithm for Object Goal Navigation in a "language-driven, zero-shot manner", where an embodied agent navigates to an arbitrarily described target object in a previously unexplored environment. The approach leverages the capabilities of Large Language Models (LLMs) for making navigational decisions by mapping the LLM's implicit knowledge about the semantic context of the environment into sequential inputs for robot motion planning.

arXiv.org

AZTR: Aerial Video Action Recognition with Auto Zoom and Temporal Reasoning

Xijun Wang, Ruiqi Xian, Tianrui Guan, Celso de Melo, Stephen Nogar, Aniket Bera, Dinesh Manocha

A new approach for aerial video action recognition that is designed for videos captured using UAVs and can run on edge or mobile devices. The learning-based approach uses customized auto zoom to automatically identify the human target and scale it appropriately. This makes it easier to extract key features and reduces the computational overhead.

arXiv.org

MITFAS: Mutual Information based Temporal Feature Alignment and Sampling for Aerial Video Action Recognition

Ruiqi Xian, Xijun Wang, Dinesh Manocha

This paper presents a novel approach for action recognition in UAV videos. The formulation is designed to handle occlusion and viewpoint changes caused by the movement of a UAV. The researchers use the concept of mutual information to compute and align the regions corresponding to human action or motion in the temporal domain. This enables the recognition model to learn from the key features associated with the motion.

arXiv.org

2022

AFR: An Efficient Buffering Algorithm for Cloud Robotic Systems

Yu-Ping Wang, Hao-Ning Wang, Zi-Xin Zou, Dinesh Manocha

Communication between robots and the server is a major problem for cloud robotic systems. The authors address the problem caused by data loss during such communications, and propose an efficient buffering algorithm, AFR, to solve the problem.

2022 International Conference on Intelligent Robots and Systems

D2-TPred: Discontinuous Dependency for Trajectory Prediction under Traffic Lights

Yuzhen Zhang, Wentong Wang, Weizhi Guo, Pei Lv, Mingliang Xu, Wei Chen, Dinesh Manocha

A profound understanding of inter-agent relationships and motion behaviors is important to achieve high-quality planning when navigating in complex scenarios, especially at urban traffic intersections. This paper presents D2-TPred, a trajectory prediction approach with respect to traffic lights, which uses a spatial dynamic interaction graph (SDG) and a behavior dependency graph (BDG) to handle the problem of discontinuous dependency in the spatial-temporal space. Specifically, the SDG is used to capture spatial interactions by reconstructing sub-graphs for different agents with dynamic and changeable characteristics during each frame.

ECCV 2022

Multi-Robot Path Planning Using Medial-Axis-Based Pebble-Graph Embedding

Liang He, Zherong Pan, Kiril Solovey, Biao Jia, Dinesh Manocha

Here is a centralized algorithm for labeled, disk-shaped Multi-Robot Path Planning (MPP) in a continuous planar workspace with polygonal boundaries. The method automatically transform the continuous problem into a discrete, graph-based variant termed the pebble motion problem, which can be efficiently solved.

GAMMA Lab papers archive

MSVIPER: Improved Policy Distillation for Reinforcement-Learning-Based Robot Navigation

Aaron M. Roth, Jing Liang, Ram Sriram, Elham Tabassi, Dinesh Manocha

The paper introduces Multiple Scenario Verifiable Reinforcement Learning via Policy Extraction (MSVIPER), a new method for policy distillation to decision trees for improved robot navigation. MSVIPER learns an “expert” policy using any Reinforcement Learning (RL) technique involving learning a state-action mapping and then uses imitation learning to learn a decision-tree policy from it. MSVIPER results in efficient decision trees and can accurately mimic the behavior of the expert policy.

arXiv.org

Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition

Divya Kothandaraman, Ming Lin, Dinesh Manocha

A learning algorithm for human activity recognition in videos. The approach is designed for UAV videos, which are mainly acquired from obliquely placed dynamic cameras that contain a human actor along with background motion.

arXiv.org

WGICP: Differentiable Weighted GICP-Based Lidar Odometry

Sanghyun Son, Jing Liang, Ming Lin, Dinesh Manocha

The paper presents a novel differentiable weighted generalized iterative closest point (WGICP) method applicable to general 3D point cloud data, including that from lidar. This method builds on differentiable generalized ICP (GICP) and uses the differentiable K-Nearest Neighbors (KNN) algorithm to enhance differentiability. Our differentiable GICP algorithm provides the gradient of output pose estimation with respect to each input point. This allows the researchers to train a neural network to predict its importance, or weight, in order to estimate the correct pose.

arXiv.org

FAR: Fourier Aerial Video Recognition

Divya Kothandaraman, Tianrui Guan, Xijun Wang, Shuowen Hu, Ming Lin, Dinesh Manocha

The algorithm Fourier Activity Recognition (FAR) is presented for UAV video activity recognition. The formulation uses a novel Fourier object disentanglement method to innately separate out the human agent (which is typically small) from the background. The disentanglement technique operates in the frequency domain to characterize the extent of temporal change of spatial pixels, and exploits convolution-multiplication properties of Fourier transform to map this representation to the corresponding object-background entangled features obtained from the network. This formulation uses much fewer computations than self-attention and has been evaluated on multiple UAV datasets including UAV Human RGB, UAV Human Night, Drone Action, and NEC Drone.

Computer Vision ECCV 2022 (17th European Conference)

HTRON: Efficient Outdoor Navigation with Sparse Rewards via Heavy Tailed Adaptive Reinforce Algorithm

Kasun Weerakoon, Souradip Chakraborty, Nare Karapetyan, Adarsh Jagan Sathyamoorthy, Amrit Singh Bedi, Dinesh Manocha

An approach to improve the performance of deep reinforcement learning (DRL) based outdoor robot navigation systems that uses easy-to-design parse rewards. The authors propose a novel adaptive Heavy-Tailed Reinforce algorithm for Outdoor Navigation called HTRON. The main idea is to utilize heavy-tailed policy parametrizations which implicitly induce exploration in sparse reward settings.

arXiv.org

Decentralized multi-agent planning for maneuvering in complex scenarios

Senthil Hariharan Arul, Dinesh Manocha

Multi-agent navigation methods for autonomous robots maneuvering in complex environments. The approach considers the agent’s dynamics to compute smooth paths that avoid collisions with other agents and static and dynamic obstacles in a decentralized fashion.

Proceedings of SPIE 12113, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications IV

Multi Robot Collision Avoidance by Learning Whom to Communicate

Senthil Hariharan Arul, Amrit Singh Bedi, Dinesh Manocha

Agents in decentralized multi-agent navigation lack the world knowledge to make safe and (near-)optimal plans reliably. They base their decisions on their neighbors' observable states, which hide the neighbors' navigation intent. Here, the authors propose augmenting decentralized navigation with inter-agent communication to improve their performance and aid an agent in making sound navigation decisions. They present a reinforcement learning method for multi-agent collision avoidance using selective inter-agent communication.

Cornell University arXiv.org

GrASPE: Graph based Multimodal Fusion for Robot Navigation in Unstructured Outdoor Environments

Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Jing Liang, Tianrui Guan, Utsav Patel, Dinesh Manocha

A trajectory traversability estimation and planning algorithm for robot navigation in complex outdoor environments. The algorithm incorporates multimodal sensory inputs from an RGB camera, 3D LiDAR, and robot’s odometry sensor to train a prediction model to estimate candidate trajectories’ success probabilities based on partially reliable multi-modal sensor observations.

arXiv.org

RTAW: An Attention Inspired Reinforcement Learning Method for Multi-Robot Task Allocation in Warehouse Environments

Aakriti Agrawal, Amrit Singh Bedi, Dinesh Manocha

Here is a reinforcement learning-based algorithm for multi-robot task allocation problem in warehouse environments. It is formulated as a Markov Decision Process and solves via a novel deep multi-agent reinforcement learning method called RTAW with attention inspired policy architecture.

arXiv.org

DC-MRTA: Decentralized Multi-Robot Task Allocation and Navigation in Complex Environments

Aakriti Agrawal, Senthil Hariharan, Amrit Singh Bedi, Dinesh Manocha

This is a reinforcement learning (RL) based task allocation and decentralized navigation algorithm for mobile robots in warehouse environments. The approach is designed for scenarios in which multiple robots are used to perform various pick up and delivery tasks.

arXiv.org

Image-Goal Navigation in Complex Environments via Modular Learning

Qiaoyun Wu, Jun Wang, Jing Liang, Xiaoxi Gong, Dinesh Manocha

The authors use a four-module approach for image-goal navigation, where an agent navigates with a goal image rather than accurate target information, which is more challenging. The work decouples the learning of navigation goal planning, collision avoidance, and navigation ending prediction, enabling more concentrated learning of each part.

IEEE Robotics and Automation Letters

Sim-to-Real Strategy for Spatially Aware Robot Navigation in Uneven Outdoor Environments

Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Dinesh Manocha

Deep Reinforcement Learning (DRL) is hugely successful due to the availability of realistic simulated environments. However, performance degradation during simulation to real-world transfer still remains a challenging problem for the policies trained in simulated environments. To close this sim-to-real gap, we present a novel hybrid architecture that utilizes an intermediate output from a fully trained attention DRL policy as a navigation cost map for outdoor navigation.

arXiv.org

AdaptiveON: Adaptive Outdoor Navigation Method for Stable and Reliable Actions

Jing Liang, Kasun Weerakoon, Tianrui Guan, Nare Karapetyan, Dinesh Manocha

A novel outdoor navigation algorithm to generate stable and efficient actions to navigate a robot to the goal. The authors use a multi-stage training pipeline to show their model produces policies that result in stable and reliable robot navigation on complex terrains. Based on the Proximal Policy Optimization (PPO) algorithm, they developed a novel method to achieve multiple capabilities for outdoor navigation tasks, namely: alleviating the robot’s drifting, keeping the robot stable on bumpy terrains, avoiding climbing on hills with steep elevation changes, and collision avoidance.

IEEE Robotics and Automation Letters

Game-Theoretic Planning for Autonomous Driving among Risk-Aware Human Drivers

Rohan Chandra, Mingyu Wang, Mac Schwager, Dinesh Manocha

A novel approach for risk-aware planning with human agents in multi-agent traffic scenarios. The approach takes into account the wide range of human driver behaviors on the road, from aggressive maneuvers like speeding and overtaking, to conservative traits like driving slowly and conforming to the right-most lane.

arXiv.org

TerraPN: Unstructured Terrain Navigation using Online Self-Supervised Learning

Adarsh Jagan Sathyamoorthy, Kasun Weerakoon, Tianrui Guan, Jing Liang, Dinesh Manocha

TerraPN is a novel method that learns the surface properties (traction, bumpiness, deformability, etc.) of complex outdoor terrains directly from robot-terrain interactions through self-supervised learning, and uses it for autonomous robot navigation.

2022 International Conference on Intelligent Robots and Systems

Fourier Disentangled Space-Time Attention for Aerial Video Recognition

Divya Kothandaraman, Tianrui Guan, Xijun Wang, Sean Hu, Ming Lin, Dinesh Manocha

Describes the new Fourier Activity Recognition computer vision algorithm for UAV video activity recognition. The formulation uses a novel Fourier object disentanglement method to innately separate out the human agent (which is typically small) from the background.

arXiv.org

SelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learning

Jaehoon Choi, Dongki Jung, Yonghan Lee, Deokhwa Kim, Dinesh Manocha, Donghwan Lee

A learning algorithm that leverages monocular simultaneous localization and mapping (SLAM) with proprioceptive sensors. Such monocular SLAM systems can provide metrically scaled camera poses.

arXiv.org

GAMEOPT: Optimal Real-time Multi-Agent Planning and Control for Dynamic Intersections

Nilesh Suriyarachchi, Faizan Tariq, Christos Mavridis, John Baras, Dinesh Manocha

Intersections without traffic signals are the sites of 40% of all crashes, 50% of serious collisions, and 20% of fatalities. When motorists approach these intersections, which often have “four-way” stop signs, and feature “unprotected” left turns, they must make choices. If two vehicles approach at approximately the same time, or a driver decides not to stop or wait their turn, safety is at issue. In the future, autonomous vehicles equipped with vehicle-to-infrastructure (V2I) communication may be able to provide solutions for handling these scenarios. Such vehicles, called connected autonomous vehicles (CAVs), will be able to provide multiple sensing and actuation points in the traffic flow. GAMEOPT is a novel hybrid approach to cooperative intersection control for dynamic, multi-lane, unsignalized intersections.

arXiv.org

2021

An Intelligent Self-driving Truck System for Highway Transportation

Dawei Wang, Lingping Gao, Ziquan Lan, Wei Li, Jiaping Ren, Jiahui Zhang, Peng Zhang, Pei Zhou, Shengao Wang, Jia Pan, Dinesh Manocha, Ruigang Yang

The paper introduces an intelligent self-driving truck system that includes a traffic simulation module for generating realistic traffic flow; a high-fidelity truck model designed and evaluated for mimicking real truck response; and an intelligent planning module with a learning-based decision making algorithm and multi-mode trajectory planner that takes into account the truck’s constraints, road slope changes, and surrounding traffic flow.

arXiv.org

COVID surveillance robot: Monitoring social distancing constraints in indoor scenarios

Adarsh Jagan Sathyamoorthy, Utsav Patel, Moumita Paul, Yash Savle, Dinesh Manocha

A method to automatically detect pairs of humans in a crowded scenario and determine whether they are within about two meters of space of each other using an autonomous mobile robot equipped with closed-circuit television (CCTV) cameras.

PLOS One

Robust 2D/3D Vehicle Parsing in Arbitrary Camera Views for CVIS

Hui Miao, Feixiang Lu, Zongdai Liu, Liangjun Zhang, Dinesh Manocha and Bin Zhou

A new approach to robustly detect and perceive vehicles in different camera views as part of a cooperative vehicle-infrastructure system (CVIS).

ICCV 2021, Computer Vision Foundation

BoMuDANet: Unsupervised Adaptation for Visual Scene Understanding in Unstructured Driving Environments

Divya Kothandaraman, Rohan Chandra, Dinesh Manocha

An unsupervised adaptation approach for visual scene understanding in unstructured traffic environments.

ICCV 2021 workshop paper, Computer Vision Foundation

TERP: Reliable Planning in Uneven Outdoor Environments using Deep Reinforcement Learning

Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Utsav Patel, Dinesh Manocha

This new method for reliable robot navigation in uneven outdoor terrains employs a novel fully-trained Deep Reinforcement Learning (DRL) network that uses elevation maps of the environment, robot pose, and goal as inputs to compute an attention mask of the environment.

arXiv.org

TTM: Terrain Traversability Mapping for Autonomous Excavator Navigation in Unstructured Environments

Tianrui Guan, Zhenpeng He, Dinesh Manocha, Liangjun Zhang

Terrain Traversability Mapping (TTM) is a real-time mapping approach for terrain traversability estimation and path planning for autonomous excavators in an unstructured environment.

arXiv.org

METEOR: A Massive Dense & Heterogeneous Behavior Dataset for Autonomous Driving

Rohan Chandra, Mridul Mahajan, Rahul Kala, Rishitha Palugulla, Chandrababu Naidu, Alok Jain, Dinesh Manocha

METEOR is a new, complex traffic dataset that captures traffic patterns in unstructured scenarios in India. METEOR consists of more than 1000 one-minute video clips, over 2 million annotated frames with ego-vehicle trajectories, and more than 13 million bounding boxes for surrounding vehicles or traffic agents. METEOR is a unique dataset in terms of capturing the heterogeneity of microscopic and macroscopic traffic characteristics.

arXiv.org

MotionHint: Self-Supervised Monocular Visual Odometry with Motion Constraints

Cong Wang, Yu-Ping Wang, Dinesh Manocha

MotionHint is a self-supervised algorithm for monocular visual odometry (VO) that takes motion constraints into account.

ADS Labs, Harvard

Joint Search of Optimal Topology and Trajectory for Planar Linkagesg

Zherong Pan1, Min Liu, Xifeng Gao, Dinesh Manocha

A pivotal challenge faced by a robot designer is to determine the “optimal” design to accomplish a given task. This paper addresses the problem of computational task-driven, design optimization for planar linkages.

arXiv.org

GamePlan: Game-Theoretic Multi-Agent Planning with Human Drivers at Intersections, Roundabouts, and Merging

Rohan Chandra, Dinesh Manocha

GamePlan is a new method for multi-agent planning involving human drivers and autonomous vehicles (AVs) for unsignaled intersections and roundabouts. It also is designed for merging situations.

arXiv.org

CoMet: Modeling Group Cohesion for Socially Compliant Robot Navigation in Crowded Scenes

Adarsh Jagan Sathyamoorthy, Utsav Patel, Moumita Paul, Nithish K Sanjeev Kumar, Yash Savle, Dinesh Manocha

CoMet is a new social navigation algorithm to compute a group's cohesion and use that to improve a robot's navigation in crowded scenes.

SAO/NASA Astrophysics Data System

Hierarchical Image-Goal Navigation in Real Crowded Scenarios

Qiaoyun Wu, Jun Wang, Jing Liang, Xiaoxi Gong, Dinesh Manocha

This work studies the problem of image-goal navigation, which entails guiding robots with noisy sensors and controls through real crowded environments.

arXiv.org

OF-VO: Efficient Navigation among Pedestrians Using Commodity Sensors

Jing Liang, Yi-Ling Qiao, Tianrui Guan, Dinesh Manocha

A modified velocity-obstacle (VO) algorithm that uses probabilistic partial observations of the environment to compute velocities and navigate a robot to a target. The system uses commodity visual sensors, including a mono-camera and a 2D Lidar, to explicitly predict the velocities and positions of surrounding obstacles through optical flow estimation, object detection, and sensor fusion.

IEEE Robotics and Automation Letters

M3DETR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformers

Tianrui Guan, Jun Wang, Shiyi Lan, Rohan Chandra, Zuxuan Wu, Larry Davis, Dinesh Manocha

M3DETR is a new architecture for 3D object detection. It combines different point cloud representations (raw, voxels, bird-eye view) with different feature scales based on multi-scale feature pyramids. Supplemental materials here.

arXiv.org; also presented at WACV 2022 (IEEE Computer Vision Society Winter Conference on Applications of Computer Vision)

XAI-N: Sensor-based Robot Navigation using Expert Policies and Decision Trees

Aaron Roth, Jin Liang, Dinesh Manocha

A novel sensor-based learning navigation algorithm to compute a collision-free trajectory for a robot in dense and dynamic environments with moving obstacles or targets.

arXiv.org

Reinforcement Learning-Based Visual Navigation With Information-Theoretic Regularization

Qiaoyun Wu, Kai Xu, Jun Wang, Mingliang Xu, Xiaoxi Gong, Dinesh Manocha

To enhance the cross-target and cross-scene generalization of target-driven visual navigation based on deep reinforcement learning (RL), the authors introduce an information-theoretic regularization term into the RL objective. The regularization maximizes the mutual information between navigation actions and visual observation transforms of an agent, thus promoting more informed navigation decisions.

IEEE Robotics and Automation Letters

B-GAP: Behavior-Guided Action Prediction and Navigation for Autonomous Driving

Angelos Mavrogiannis, Rohan Chandra, Dinesh Manocha

An algorithm for behaviorally-guided action prediction and local navigation for autonomous driving in dense traffic scenarios.

IEEE Robotics and Automation Letters 2022

GANav: Group-wise Attention for Classifying Navigable Regions in Unstructured Outdoor Environments

Tianrui Guan, Divya Kothandaraman, Rohan Chandra, Dinesh Manocha

A learning-based method for identifying safe and navigable regions in off-road terrains and unstructured environments from RGB images. The approach consists of classifying groups of terrain classes based on their navigability levels using coarse-grained semantic segmentation.

arXiv.org

V-RVO: Decentralized Multi-Agent Collision Avoidance using Voronoi Diagrams and Reciprocal Velocity Obstacles

Senthil Hariharan Arul, Dinesh Manocha

The authors present a decentralized collision avoidance method for dense environments that is based on buffered Voronoi cells (BVC) and reciprocal velocity obstacles (RVO). The approach is designed for scenarios with large numbers of close proximity agents and provides passive-friendly collision avoidance guarantees.

2021 IEEE/RSJ International Conference on Intelligent Robots and Systems

Multi-Agent Ergodic Coverage in Urban Environments

Shivang Patel, Senthil Hariharan, Pranav Dhulipala, Ming C Lin1, Dinesh Manocha, Huan Xu, Michael Otte

An important aspect of dynamic urban coverage is how building collision avoidance is incorporated into the overall coverage mission. We consider a multi-agent urban dynamic coverage problem in which a team of flying agents uses downward facing cameras to observe the street-level environment outside of buildings.

GAMMA Lab website; presented at ICRA 2021

CoMet: Modeling Group Cohesion for Socially Compliant Robot Navigation in Crowded Scenes

Adarsh Jagan Sathyamoorthy, Utsav Patel, Moumita Paul, Nithish K Sanjeev Kumar, Yash Savle, Dinesh Manocha

CoMet is a novel approach for computing a group’s cohesion and using that to improve a robot’s navigation in crowded scenes. This approach uses a novel cohesion-metric that builds on prior work in social psychology.

GAMMA Lab website; presented at IROS 2020

2020

DWA-RL: Dynamically Feasible Deep Reinforcement Learning Policy for Robot Navigation in Dense Mobile Crowds

Utsav Patel, Nithish Kumar, Adarsh Jagan Sathyamoorthy, Dinesh Manocha

A novel Deep Reinforcement Learning (DRL) based policy for mobile robot navigation in dynamic environments that computes dynamically feasible and spatially aware robot velocities. The method addresses two primary issues associated with the Dynamic Window Approach (DWA) and DRL-based navigation policies and solves them by using the benefits of one method to fix the issues of the other.

arXiv.org

SwarmCCO: Probabilistic Reactive Collision Avoidance for Quadrotor Swarms under Uncertainty

Senthil Hariharan Arul, Dinesh Manocha

Decentralized collision avoidance algorithms for quadrotor swarms operating under uncertain state estimation. The approach exploits the differential flatness property and feed-forward linearization to approximate the quadrotor dynamics and reciprocal collision avoidance.

arXiv.org

Take an Emotion Walk: Perceiving Emotions from Gaits Using Hierarchical Attention Pooling and Affective Mapping

Uttaran Bhattacharya, Christian Roncal, Trisha Mittal, Rohan Chandra, Kyra Kapsaskis, Kurt Gray, Aniket Bera, Dinesh Manocha

Presents an autoencoder-based semi-supervised approach to classify perceived human emotions from walking styles obtained from videos or motion-captured data and represented as sequences of 3D poses.

Computer Vision--ECCV 2020 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X

ORBBuf: A Robust Buffering Method for Collaborative Visual SLAM

Yu-Ping Wang, Zi-Xin Zou, Cong Wang, Yue-Jiang Dong, Lei Qiao, Dinesh Manocha

Presents a novel buffering method, ORBBuf, to reduce the impact of data loss on collaborative visual SLAM systems.

arXiv.org

Dynamically Feasible Deep Reinforcement Learning Policy for Robot Navigation in Dense Mobile Crowds

Utsav Patel, Nithish Kumar, Adarsh Jagan Sathyamoorthy and Dinesh Manocha

A novel Deep Reinforcement Learning (DRL) based policy for mobile robot navigation in dynamic environments that computes dynamically feasible and spatially aware robot velocities.

arXiv.org

Towards Target-Driven Visual Navigation in Indoor Scenes via Generative Imitation Learning

Qiaoyun Wu, Xiaoxi Gong, Kai Xu, Dinesh Manocha, Jingxuan Dong, Jun Wang

A target-driven navigation system to improve mapless visual navigation for robots in indoor scenes.

arXiv.org

COVID-Robot: Monitoring Social Distancing Constraints in Crowded Scenarios

Adarsh Jagan Sathyamoorthy, Utsav Patel, Yash Ajay Savle, Moumita Paul, Dinesh Manocha

Maintaining social distancing norms between humans has become an indispensable precaution to slow dowthe transmission of COVID-19. The paper presents a novel method to automatically detect pairs of humans in a crowded scenario who are not adhering to the 6-foot social distance constraint. The approach makes no assumption about crowd density or pedestrian walking directions. A mobile robot is fitted with commodity sensors, namely an RGB-D camera and a 2-D lidar to perform collision-free navigation in a crowd and estimate the distance between all detected individuals in the camera’s field of view. The robot is also equipped with a thermal camera that wirelessly transmits thermal images to security/healthcare personnel who monitor if any individual exhibits a higher than normal temperature. In indoor scenarios, the mobile robot also can be combined with static mounted CCTV cameras to further improve the performance of the number of social distancing breaches detected. The robot can accurately pursue walking pedestrians etc.

arXiv.org

Autonomous Social Distancing in Urban Environments using a Quadruped Robot

Tingxiang Fan, Zhiming Che∗, Xuan Zhao, Jing Liang, Cong Shen, Dinesh Manocha, Jia Pan, Wei Zhang

The paper presents a fully autonomous surveillance robot based on a quadruped platform that can promote social distancing in complex urban environments. The researchers mount multiple cameras and a 3D LiDAR on the legged robot. The robot then uses an onboard real-time social distancing detection system to track nearby pedestrian groups. Next, the robot uses a crowd-aware navigation algorithm to move freely in highly dynamic scenarios. The robot finally uses a crowd-aware routing algorithm to effectively promote social distancing by using human-friendly verbal cues to send suggestions to over-crowded pedestrians.

arXiv.org

Accurate High Fidelity Simulations for Training Robot Navigation Policies for Dense Crowds using Deep Reinforcement Learning

Jin Liang, Ustav Patel, Adarsh Sathyamoorthy, Dinesh Manocha

A novel high fidelity 3-D simulator that significantly reduces the sim-to-real gap in mobile robotics training collision avoidance policies based on Deep Reinforcement Learning for dense crowd scenarios.

sim2real.github.io

AutoTrajectory: Label-free Trajectory Extraction and Prediction from Videos using Dynamic Points

Yuexin Ma, Xinge Zhu, Xinjing Cheng, Ruigang Yang, Jiming Liu, Dinesh Manocha

For intelligent agents like robots and autonomous vehicles, it is crucial to be able to forecast neighboring traffic-agents’ future trajectories for navigation and planning applications. Current methods for trajectory prediction operate in super-vised manners, and therefore require vast quantities of corresponding ground truth data for training. The paper presents a novel, label-free algorithm, AutoTrajectory, for trajectory extraction and prediction to use raw videos directly.

arXiv.org

DenseCAvoid: Real-time Navigation in Dense Crowds using Anticipatory Behaviors

Adarsh Jagan Sathyamoorthy, Jing Liang, Utsav Patel, Tianrui Guan, Rohan Chandra, Dinesh Manocha

A navigation algorithm for navigating a robot through dense crowds and avoiding collisions by anticipating pedestrian behaviors. The formulation uses visual sensors and a pedestrian trajectory prediction algorithm to track pedestrians in a set of input frames and provide bounding boxes that extrapolate the pedestrian positions in a future time. This hybrid approach combines trajectory prediction with a Deep Reinforcement Learning-based collision avoidance method to train a policy to generate smoother,safer, and more robust trajectories during run-time.

arXiv.org

Efficient Multi-Agent Motion Planning in Continuous Workspaces Using Medial-Axis-Based Swap Graphs

Liang He, Zherong Pan, Biao Jia, Dinesh Manocha

An algorithm for homogeneous, labeled, and disk-shaped multi-agent motion planning in continuous workspaces with arbitrarily-shaped obstacles.

arXiv.org

ProxEmo: Gait-based Emotion Learning and Multi-view Proxemic Fusion for Socially-Aware Robot Navigation

Venkatraman Narayanan, Bala Murali Manoghar, Vishnu Sashank Dorbala, Dinesh Manocha, Aniket Bera

ProxEmo is an end-to-end emotion prediction algorithm for socially aware robot navigation among pedestrians. This approach predicts the perceived emotions of a pedestrian from walking gaits, which is then used for emotion-guided navigation taking into account social and proxemic constraints. To classify emotions, ProxEmo uses a multi-view skeletongraph convolution-based model that works on a commodity camera mounted onto a moving robot. The emotion recognition is integrated into a mapless navigation scheme and makes no assumptions about the environment of pedestrian motion.

arXiv.org

Frozone: Freezing-Free, Pedestrian-Friendly Navigation in Human Crowds

Adarsh Jagan Sathyamoorthy, Utsav Patel, Tianrui Guan, Dinesh Manocha

Frozone is a novel algorithm to deal with the Freezing Robot Problem that arises when a robot navigates through dense scenarios and crowds. The method senses and explicitly predicts the trajectories of pedestrians and constructs a Potential Freezing Zone (PFZ); a spatial zone where the robot could freeze or be obtrusive to humans. The formulation computes a deviation velocity to avoid the PFZ, which also accounts for social constraints. Furthermore, Frozone is designed for robots equipped with sensors with a limited sensing range and field of view. The researchers ensure that the robot’s deviation is bounded, thus avoiding sudden angular motion which could lead to the loss of perception data of the surrounding obstacles.

arXiv.org

Learning Resilient Behaviors for Navigation Under Uncertainty

Tingxiang Fan, Pinxin Long, Wenxi Liu, Jia Pan, Ruigang Yang, Dinesh Manocha

An uncertainty-aware navigation approach for mobile robots that introduces a predictor to model the environmental uncertainty. In addition, the paper proposes an uncertainty-aware navigation network to learn resilient behaviors in prior unknown environments.

2020 International Conference on Robotics and Automation

CrowdSteer: Realtime Collision Avoidance for Mobile Robots in Dense Crowds using Implicit Multi-sensor Fusion and Deep Reinforcement Learning

Jing Liang, Utsav Patel, Adarsh Jagan Sathyamoorthy, Dinesh Manocha

CrowdSteer is a learning-based collision avoidance algorithm for mobile robots operating in dense and crowded environments.

19th International Conference on Autonomous Agents and Multiagent Systems

OF-VO: Reliable Navigation among Pedestrians Using Commodity Sensors

Jing Liang, Yi-Ling Qiao, Dinesh Manocha

An algorithm that uses commodity visual sensors, including RGB-D cameras and a2D lidar for safe navigation of a mobile robot among pedestrians. The algorithm explicitly predicts the velocities and positions of surrounding obstacles through optical flow estimation and object detection.

arXiv.org

3D-OGSE: Online Smooth Trajectory Generation for Quadrotors using Generalized Shape Expansion in Unknown 3D Environments

Vrushabh Zinage, Senthil Hariharan Arul, Dinesh Manocha

An online motion planning algorithm for generating smooth, collision-free trajectories for quadrotors operating in an unknown, cluttered 3D environment.

arXiv.org

How are You Feeling? Multimodal Emotion Learning for Socially-Assistive Robot Navigation

Aniket Bera, Tanmay Randhavane, Rohan Prinja, Kyra Kapsaskis, Austin Wang, Kurt Gray, Dinesh Manocha

A real-time algorithm that learns the emotion state of the pedestrians to perform socially-aware navigation. The robot learns pedestrians’ emotions and their proxemic constraints to improve both social comfort and navigation.

15th IEEE International Conference on Automatic Face and Gesture Recognition

HMPO: Human Motion Prediction in Occluded Environments for Safe Motion Planning

Jae Sung Park, Dinesh Manocha

A new approach to generate collision-free trajectories for a robot operating in close proximity with a human obstacle in an occluded environment.

arXiv.org

Deep Differentiable Grasp Planner for High-DOF Grippers

Min Liu, Zherong Pan, Kai Xu, Kanishka Ganguly, Dinesh Manocha

An end-to-end algorithm for training deep neural networks to grasp novel objects.

arXiv.org

Grasping Fragile Objects using a Stress-Minimization Metric

Zherong Pan, Xifeng Gao, Dinesh Manocha

A new stress-minimization metric to generate optimal grasps for brittle and fragile objects.

github.io

2019

Reinforcement Learning-Based Visual Navigation with Information-Theoretic Regularization

Qiaoyun Wu, Kai Xu, Jun Wang, Mingliang Xu, Dinesh Manocha

The authors integrate an information-theoretic regularization into a deep reinforcement learning framework for the target-driven task of visual navigation in robotics. This is achieved by first learning to generate a next observation from a current observation and a navigation target, then planning an action toward the target based on the generated observation and the current observation.

arxiv.org

New Formulation of Mixed-Integer Conic Programming for Globally Optimal Grasp Planning

Min Liu, Zherong Pan, Kai Xu, and Dinesh Manocha

Classical robotics research presents the first grasp planning algorithm to compute globally optimal gripper pose that maximizes a grasp metric.

arXiv.org

DCAD: Decentralized Collision Avoidance with Dynamics Constraints for Agile Quadrotor Swarms

Senthil Hariharan Arul, Dinesh Manocha

A new algorithm for decentralized collision avoidance for quadrotor swarm navigation in dense environments with static and dynamic obstacles.

arXiv.org

3-D Deformable Object Manipulation using Deep Neural Networks

Zhe Hu, Tao Han, Peigen Sun, Jia Pan, Dinesh Manocha

Deep neural network-based controller to servo-control position and shape of deformable objects with unknown deformation properties.

IEEE Robotics and Automation Letters

DeepMNavigate: Deep Reinforced Multi-Robot Navigation Unifying Local & Global Collision Avoidance

Qingyang Tan, Tingxiang Fan, Jia Pan, Dinesh Manocha

A novel algorithm (DeepMNavigate) for global multi-agent navigation in dense scenarios using deep reinforcement learning. | Watch a video about DeepMNavigate |

qytan.com

GraphRQI: Classifying Driver Behaviors Using Graph Spectrums

Rohan Chandra, Uttaran Bhattacharya, Trisha Mittal, Xiaoyu Li, Aniket Bera, Dinesh Manocha

The GraphRQI algorithm identifies driver behaviors from road agent trajectories. It is 25 percent more accurate over prior behavior classification algorithms for autonomous vehicles.  | Watch a video about GraphRQI |

arxiv.org

Realtime Simulation of Thin-Shell Deformable Materials using CNN-Based Mesh Embedding

Qingyang Tan, Zherong Pan, Lin Gao, and Dinesh Manocha

A new method bridges the gap between mesh embedding and physical simulation for efficient dynamic models of clothes. The key technique is a graph-based convolutional neural network (CNN) defined on meshes with arbitrary topologies, and a new mesh embedding approach based on physics-inspired loss term. After training, the learned simulator runs10–100 times faster and the accuracy is high enough for robot manipulation tasks. | Watch a video about this method |

arxiv.org

2023

PiP-X: Online feedback motion planning/replanning in dynamic environments using invariant funnels

Mohamed Khalid Jaffar, Michael Otte

The paper proposes an online single-query sampling-based motion re-planning algorithm using finite-time invariant sets, commonly referred to as “funnels.” The authors combine concepts from nonlinear systems analysis, sampling-based techniques, and graph-search methods to create a single framework that enables feedback motion re-planning for any general nonlinear dynamical system in dynamic workspaces.

International Journal of Robotics Research

Adaptive Exploration-Exploitation Active Learning of Gaussian Processes

George Kontoudis, Michael Otte

Active Learning of Gaussian process (GP) surrogates is an efficient way to model unknown environments in various applications. In this paper, we propose an adaptive exploration-exploitation active learning method (ALX) that can be executed rapidly to facilitate real-time decision making. For the exploration phase, we formulate an acquisition function that maximizes the approximated, expected Fisher information. For the exploitation phase, we employ a closed-form acquisition function that maximizes the total expected variance reduction of the search space. The determination of each phase is established with an exploration condition that measures the predictive accuracy of GP surrogates. Extensive numerical experiments in multiple input spaces validate the efficiency of our method.

preprint on Kontoudis website

A Heuristic-Guided Dynamical Multi-Rover Motion Planning Framework for Planetary Surface Missions

Sharan Nayak;, Michael Paton, Michael Otte

The paper presents a heuristic-guided multi-robot motion planning framework that solves the problem of n dynamical agents visiting m unlabeled targets in a partially known environment for planetary surface missions without solving the two-point boundary value problem.

2023 IEEE Robotics and Automation Letters

Low-level controller in response to changes in quadrotor dynamics

Jae-Kyung Cho, Chan Kim, Mohamed Khalid M Jaffa, Michael Otte, Seong-Woo Kim

The paper focuses on training a low-level controller, which operates in response to dynamics-changes without prior knowledge or fine-tuning of the parameters, using reinforcement learning. The authors randomize the dynamics of quadrotors in the simulator and train the policy based on dynamics information extracted from the state-action history through recurrent neural networks (RNNs). They propose a novel structure with excellent performance, verified by testing experiments that stabilize quadrotors with different dynamics.

2023 IEEE International Conference on Robotics and Automation (ICRA)

Path-Based Sensors: Will the Knowledge of Correlation in Random Variables Accelerate Information Gathering?

Alkesh Srivastava, George Kontoudis, Donald Sofge, Michael Otte

In some missions it is typical for target elements to be in close proximity to hazardous factors that hinder the information-gathering process. In this study, we examine a similar scenario and conduct experiments to determine if additional knowledge about the correlation between hazards and targets improves the efficiency of information gathering. To incorporate this knowledge, the researchers utilize a Bayesian network representation of domain knowledge and develop an algorithm based on this representation. An empirical investigation reveals that such additional information on correlation is beneficial only in environments with moderate hazard lethality, suggesting that while knowledge of correlation helps, further research and development is necessary for optimal outcomes.

arXiv.org

Closed-Form Active Learning using Expected Variance Reduction of Gaussian Process Surrogates for Adaptive Sampling

George Kontoudis, Michael Otte

The authors propose an active learning method of expected variance reduction with Gaussian process (GP) surrogates using a closed-form gradient. The use of closed-form gradient leads the optimization to find better solutions with reduced computations. They derive the closed-form gradient for active learning Cohn (ALC) using GP surrogates that are formed with the separable squared exponential covariance function.

researchgate.net

Control and Dynamic Motion Planning for a Hybrid Air-Underwater Quadrotor: Minimizing Energy Use in a Flooded Cave Environment

Ilya Semenov, Robert Brown, Michael Otte

A dynamic path planning algorithm to navigate an amphibious rotor craft through a concave time-invariant obstacle field while attempting to minimize energy usage.

arXiv.org

2022

PiP-X: Funnel-Based Online Feedback Motion Planning/Replanning in Dynamic Environments

Mohamed Jaffar, Michael Otte

The authors introduce an online single-query sampling-based feedback motion re-planning algorithm using finite-time invariant sets, “funnels". They combine concepts from nonlinear systems analysis, sampling-based motion planning, and graph-search methods to create a single framework that enables feedback motion planning/replanning for general nonlinear dynamical systems in a dynamic workspace. They introduce a novel graph data structure to represent a network of volumetric funnels, enabling the use of quick graph-replanning techniques.

WAFR 2022: Algorithmic Foundations of Robotics XV

Distributed Multi-Robot Information Gathering using Path-Based Sensors in Entropy-Weighted Voronoi Regions

Alkesh Srivastava, George Kontoudis, Donald Sofge, Michael Otte

A distributed information-gathering algorithm for multi-robot systems that use multiple path-based sensors to infer the locations of hazards within the environment.

researchgate.net

Decentralized Robot Swarm Clustering: Adding Resilience to Malicious Masquerade Attacks

Mitali Gandhe, Michael Otte

The authors compare the resilience of four distributed robot swarm clustering algorithms to masquerade attacks launched from malicious robots within the swarm.

15th International Workshop on the Algorithmic Foundations of Robotics (WAFR 2022)

2023

Autonomous Aerial Search and Revisit Behavior for Communication Limited Environments

Wei Cui, Animesh Shastry, Derek Paley, Stephen Nogar

An autonomous search and revisit strategy for multi-agent unmanned aerial systems (UAS), specifically tailored for long-range operations where communication may be limited or denied.

CDCL paper

Output Feedback Formation Control of a School of Robotic Fish with Artificial Lateral Line Sensing

Artur Wolek, Derek Paley

An estimation and control framework to stabilize the parallel motion of a school of robotic fish using sensory feedback. Each robot is modeled as a constant speed, planar, self-propelled particle that produces a flowfield according to a dipole potential flow model. An artificial lateral line system senses pressure fluctuations at several locations along each robot’s body. The equations of motion and measurement model are formulated in a path frame that translates and rotates with each robot’s position and velocity, respectively. A particle filter estimates the relative position and heading of other nearby robots. The resulting estimate drives a Lyapunov-based formation controller to synchronize the headings of the robots and achieve a parallel formation. Numerical simulations illustrate the proposed approach.

CDCL paper; IROS 2023

A lionfish-inspired predation strategy in planar structured environments

Anthony Thompson, Ashley Peterson, Matthew McHenry, Derek Paley

This paper investigates a pursuit-evasion game with a single pursuer and evader in a bounded environment, inspired by observations of predation attempts by lionfish (Pterois sp.). The pursuer tracks the evader with a pure pursuit strategy while using an additional bioinspired tactic to trap the evader, i.e. minimize the evader's escape routes. Specifically, the pursuer employs symmetric appendages inspired by the large pectoral fins of lionfish, but this expansion increases its drag and therefore its work to capture the evader. The evader employs a bioinspired randomly-directed escape strategy to avoid capture and collisions with the boundary. The authors investigate the trade-off between minimizing the work to capture the evader and minimizing the evader's escape routes. By using the pursuer's expected work to capture as a cost function, they determine when the pursuer should expand its appendages as a function of the relative distance to the evader and the evader's proximity to the boundary. Visualizing the pursuer's expected work to capture everywhere in the bounded domain, yields additional insights about optimal pursuit trajectories and illustrates the role of the boundary in predator-prey interactions.

Bioinspiration & Biomimetics

Data-Driven Prediction of Urban Micromobility: A Study of Dockless Electric Scooters [Applications of Control]

Debdipta Goswami, Ari Riggins, Derek Paley

E-scooters are the latest development for urban micromobility solutions. Due to their small sizes and effectiveness as a first-/last-mile solution, dockless e-scooters are increasingly popular in U.S. cities. The lack of widespread statistical studies of e-scooter mobility patterns forces many e-scooter rental companies to resort to empirical methods for deploying the vehicles. This article proposes a data-driven method to predict e-scooter mobility in urban areas. Scooter trip origins and destinations are partitioned by geographical region, and the trip origin volume is predicted as a time series using an echo-state network coupled with an ensemble Kalman filter. A Markov operator prediction technique is used to predict the spatiotemporal distribution of the scooter trip destinations.

IEEE Control Systems Magazine

2022

Distributed Estimation of the Pelagic Scattering Layer using a Buoyancy Controlled Robotic System

Cong Wei, Derek Paley

This paper formulates a strategy for Driftcam, an ocean-going robot system, to observe and track the motion of an ocean biological phenomenon called the pelagic scattering layer, which consists of organisms that migrate vertically in the water column once per day. Driftcam’s horizontal motion is determined by the flow field and the vertical motion is regulated by onboard buoyancy control. To observe the evolution of the scattering layer, an ensemble Kalman filter is applied to estimate organism density; the density dynamics are propagated using the Perron-Frobenius operator. Multiple Driftcam are subject to depth regulation by open-loop and closed-loop controllers; a control strategy is proposed to track the peak of the density. Numerical simulations illustrate the efficacy of this strategy and motivate ongoing and future efforts to design a coordination formation algorithm for multi-agent Driftcam system to track the motion of the scattering layer, with implications for ocean monitoring.

CDCL paper

Dynamics and Stability Analysis of a Tethered Unmanned Rotorcraft

Alexander Donkels, Johann Dauer, Derek Paley

An analysis of the dynamics and stability of a nonlinear model of a small-scale tethered rotorcraft. The authors develop a simplified model of a rotorcraft’s longitudinal dynamics and vary model parameters including tether force, trim conditions, and the horizontal wind to study the interdependence of those parameters and their impact on the model’s equilibrium points and stability. This is a preliminary step towards design of an automatic control of an unmanned rotorcraft capable of autonomous tethered landing and development of tether force control laws for the winch device.

CDCL paper

Distributed spacing control for multiple, buoyancy-controlled underwater robots

Cong Wei, Derek Paley

This paper presents a distributed coordination algorithm for multiple, buoyancy controlled underwater robots to achieve a moving formation in a shear flow.

CDCL paper

2021

Optimal control of a 2D diffusion–advection process with a team of mobile actuators under jointly optimal guidance

Sheng Cheng, Derek Paley

This paper describes an optimization framework to control a distributed parameter system (DPS) using a team of mobile actuators. The framework simultaneously seeks optimal control of the DPS and optimal guidance of the mobile actuators such that a cost function associated with both the DPS and the mobile actuators is minimized subject to the dynamics of each.

Automatica

Optimal guidance and estimation of a 2D diffusion-advection process by a team of mobile sensors

Sheng Cheng, Derek Paley

Describes an optimization framework to design guidance for a possibly heterogeneous team of multiple mobile sensors to estimate a spatiotemporal process modeled by a 2D diffusion-advection process.

arXiv.org—submitted to Automatica

Modeling Flight and Battery Dynamics of a Flapping-Gliding Unmanned Aerial Vehicle

Lena Johnson, Derek Paley, Hugh Bruck

A hybrid state-space model of a UAV capable of flapping and gliding.

AIAA Journal of Guidance, Control and Dynamics

Swarming Driftcams: a Novel Platform for Locating and Tracking Pelagic Scattering Layers

Eric Berkenpas, Charles Shepard, Rachel Suitor, Paul Zaidins, Derek Paley, Kyle Abernathy

A paper about the design, development, and field results of novel depth-controlled sensor platforms. Called the “Swarming Driftcams,” these platforms travel passively in horizontal direction with ocean currents while controlling and profiling depth. All aspects of the vehicle hardware are discussed.

CDCL paper

Burrowing Locomotion via Crack Propagation of a Bio-inspired Soft Robot

John Lathrop, Derek Paley

Presents a state-space model of worm-inspired burrowing locomotion using discrete elastic rod theory applied to a segmented body representing the worm’s body. The model is controlled by changing the intrinsic length of each segment, with several peristaltic travelling-wave gaits considered. Simulations varying gait parameters show that some travelling-wave gaits allow the worm model to make faster progress. The derived state-space model permits improved control and estimation for worm-inspired robots and empirically shows how differing travelling-wave gaits affect the speed of locomotion.

CDCL paper

Optimal control of a 2D diffusion-advection process with a team of mobile actuators under jointly optimal guidance

Sheng Cheng, Derek Paley

This paper describes an optimization framework to control a distributed parameter system using a team of mobile actuators.

Paper submitted to Automatica

Observability Measures for Space-Based Cislunar Domain Awareness

Erin Fowler, Stella Hurtt, and Derek Paley

A dynamic simulation of the cislunar environment for use in numerical analysis of various pairings of resident space objects and sensing satellites intended for cislunar space domain awareness.

CDCL paper

Modeling the Flight Dynamics and Battery Utilization of a Hybrid Flapping-Gliding UAV

Lena Johnson, Derek Paley, Hugh Bruck

A hybrid flapping-gliding dynamic model for a flapping-wing unmanned aerial vehicle. The model combines the nonlinear phugoid dynamics of gliding flight, the equations of motion of the onboard servo motors, and an equivalent circuit discharge model for the onboard battery.

AIAA Scitech 2021 Forum

2020

Optimal Guidance and Estimation of a 1D Diffusion Process by a Team of Mobile Sensors

Sheng Cheng, Derek Paley

A framework to design guidance for a team of mobile sensors to estimate a distributed parameter system modeled by a diffusion process.

CDCL paper

Distributed Control of a Planar Discrete Elastic Rod for Eel-Inspired Underwater Locomotion

William Scott, Prateek Jaya Prakash, Derek A. Paley

Many organisms achieve locomotion through undulatory traveling-wave motions, both on land and underwater. Recently, caterpillar locomotion has been modeled using the theory of planar discrete elastic rods (PDER) and open-loop control. This work considers a planar discrete elastic rod model with closed-loop control over the intrinsic material parameters of length and curvature distributed along its length. The authors introduce local curvature feedback control laws to drive the shape of the robot to a desired traveling-wave reference trajectory, where the phase of the wave is determined via a central pattern generator or via distributed control with a circulant communication topology. Through numerical simulations utilizing simple models of fluid–body interaction forces we examine undulatory eel-like swimming gaits that give rise to net forward motion. These results show promise for the design of distributed feedback control laws in modular soft robotic systems.

Chapter in Bioinspired Sensing, Actuation, and Control in Underwater Soft Robotic Systems (Springer 2021)

Tracking Performance of Model-Based Thruster Control of a Remotely Operated Underwater Vehicle

Jordan Boehm, Eric Berkenpas, Charles Shepard, Derek A. Paley

The paper compares output feedback control strategies for an underwater thruster.

IEEE Journal of Oceanic Engineering

Bioinspired pursuit with a swimming robot using feedback control of an internal rotor

Brian Free, Jinseong Lee, Derek Paley

The paper presents a state-feedback control law for steering a fish-inspired robot in a desired direction, and engaging it in pure pursuit.

Bioinspiration and Biomimetics

Cooperative Mapping and Target Search over an Unknown Occupancy Graph Using Mutual Information

Artur Wolek, Sheng Cheng, Debdipta Goswami, Derek Paley

A cooperative mapping and target-search algorithm for a team of autonomous quadrotors equipped with noisy, range-limited sensors. The algorithm can concurrently map and search an unknown urban area, while detecting and tracking a mobile ground target.

IEEE Robotics and Automation Letters

Optimal control of a 1D diffusion process with a team of mobile actuators under jointly optimal guidance

Sheng Cheng, Derek Paley

An optimization framework to control a distributed parameter system (DPS) using a team of mobile actuators. The optimization simultaneously seeks efficient guidance of the mobile actuators and effective control of the DPS such that an integrated cost function associated with both the mobile actuators and the DPS is minimized.

CDCL paper

Feedback Control of a Soft Swinging Appendage

Travis Burch, John Lathrop, William Scott, Derek Paley

A state-space description using planar discrete elastic rod theory of a soft robotic appendage with torque input at one end. The authors design a linear output feedback controller to balance the appendage in an unstable vertical configuration with an angle sensor and torque input co-located at the base.

CDCL paper

Geometric Attitude and Position Control of a Quadrotor in Wind

William Craig, Derrick Yeo, Derek Paley

The researchers use  a model of the aerodynamic interaction between the propellers and wind, paired with onboard flow sensing and feedback control, to improve the stability of quadrotors in unsteady winds with the long-term goal of enabling reliable outdoor flight in windy conditions.

AIAA Journal of Guidance, Control, and Dynamics

Orbit Design for Cislunar Space Domain Awareness

Erin Fowler, Stella Hurtt, Derek Paley

Quantitative assessments of the orbits and sensor characteristics of satellites intended for cislunar space domain awareness.

2nd IAA Conference on Space Situational Awareness

 

2019

Output Feedback Control for Lift Maximization of a Pitching Airfoil

Justin M. Lidard, Debdipta Goswami, David Snyder, Girguis Sedky, Anya Jones, Derek Paley

Unsteady aerodynamics is driving research at the interface of fluid dynamics and control theory for low Reynolds number aircraft such as micro air vehicles (MAVs). The regulation and control of unsteady behavior is crucial for maintaining the stability of an MAV, which necessitates accurate modeling of their flight surfaces. This paper describes the implementation of the Goman-Khrabrov model for flow stagnation near an actuated airfoil with a feedback-controlled pitch rate for the purpose of maximizing the time-averaged unsteady lift.

AIAA SciTech 2020 Forum

Non-Gaussian Estimation of a Potential Flow by an Actuated Lagrangian Sensor Steered to Separating Boundaries by Augmented Observability

Francis D. Lagor, Kayo Ide, Derek Paley

An architecture for estimation of a flow field using a hypothetical oceanographic vehicle that is guided along paths of high flow-field observability, a concept quantifying the informativeness of a path.

IEEE Journal of Oceanic Engineering

Mobile Sensor Networks Control: Adaptive Sampling of Spatiotemporal Processes

Derek Paley, Artur Wolek

A review of control of mobile sensor networks for environmental monitoring and other applications. Includes sensor platform dynamics and cooperative control and estimation; process modeling and estimation for both continuous and discrete models; sampling metrics and optimization, including coverage-, topology-, information-, and estimation-based metrics; and methods for task design and allocation.

Annual Review of Control, Robotics, and Autonomous Systems

Stabilization of a Quadrotor in Wind with Flow Sensing: Linear Modeling and Control for Attitude and Position Hold

William Craig, J. T. Lewis, Derek Paley

This paper describes a linear controller that has been optimized for gust rejection using CONDUITR©, based on system identification performed with CIFERR©. Additionally, a custom flow probe package was used to investigate the benefits of flow feedback for gust rejection. Experiments were performed with a 210mm quadrotor system running Cleanflight software, where flow feedback yielded improvements for both short and long time-scale gusts, particularly for the longer time-scale five second gusts when the vehicle had time to settle in the wind.

VFS Autonomous VTOL Technical Meeting and Electric VTOL Symposium

 

2019

Recognizing Hemiparetic Ankle Deficits Using Wearable Pressure Sensors

Ahmed Ramadan, Anindo Roy, Elisabeth Smela

This paper provides proof of concept for a novel method to recognize impaired push-off and foot-drop deficits in hemiparetic gait using analog pressure sensors. These data may enhance feedback from a modular ankle exoskeleton (such as Anklebot) for stroke rehabilitation.

IEEE Journal of Translational Engineering in Health and Medicine

2023

Toward a novel soft robotic system for minimally invasive interventions

Noah Barnes, Olivia Young, Adira Colton, Xiaolong Liu, Miroslaw Janowski, Dheeraj Gandhi, Ryan Sochol, Jeremy Brown, Axel Krieger

During minimally invasive surgery, surgeons maneuver tools through complex anatomies, which is difficult without the ability to control the position of the tools inside the body. A potential solution for a substantial portion of these procedures is the efficient design and control of a pneumatically actuated soft robot system. The authors designed and evaluated a system to control a steerable catheter tip. A macroscale 3D printed catheter tip was designed to have two separately pressurized channels to induce bending in two directions. A motorized hand controller was developed to allow users to control the bending angle while manually inserting the steerable tip. Preliminary characterization of two catheter tip prototypes was performed and used to map desired angle inputs into pressure commands. The integrated robotic system allowed both a novice and a skilled surgeon to position the steerable catheter tip at the location of cylindrical targets with sub-millimeter accuracy. The novice was able to reach each target within ten seconds and the skilled surgeon within five seconds on average.

International Journal of Computer Assisted Radiology and Surgery

2023

Dynamic Behavior of Bistable Shallow Arches: from Intrawell to Chaotic Motion

Michael Bonthron, Eleonora Tubaldi

Bistable shallow arches are ubiquitous in many engineering systems ranging from compliant mechanisms and biomedical stents to energy harvesters and passive fluidic controllers. In all these scenarios, the bistable states of the arch and the sudden transition between them via snap-through instability are harnessed. However, bistable arches have been traditionally studied and characterized by triggering snap-through instability using quasi-static forces. Here, the authors analytically examine the effect of oscillatory loads and investigate the dynamic behavior of bistable arches spanning from intrawell motion to periodic and chaotic interwell motion.

ASME Journal of Applied Mechanics

Machine Intelligence Accelerated Design of Conductive MXene Aerogels with Programmable Properties

Snehi Shrestha, Kieran James Barvenik, Tianle Chen, Haochen Yang, Yang Li, Meera Muthachi Kesavan, Joshua M. Little, Eleonora Tubaldi, Po-Yen Chen

An integrated workflow is developed to combine collaborative robotics with machine learning to accelerate the design of conductive aerogels with programmable electrical and mechanical properties. The model-suggested conductive aerogels with high electrical conductivity, customized compression resilience, and pressure insensitivity allow for compression-stable Joule heating for wearable thermal management. The fusion of robotics-accelerated experimentation, machine intelligence, and simulation tools expedites the tailored design and scalable production of conductive aerogels.

Research Square

Investigation of the Buckling Instability in Free Thin-Shell Domes

Kieran J Barvenik, Zachary Coogan, Matteo Pezzulla, Eleonora Tubaldi

Shell buckling presents numerous opportunities to program fast and reversible reconfigurations into multifunctional devices. Applications such as soft actuators, locomotion systems, and biomedical instruments can leverage the diverse deformation properties of shell buckling mode shapes. Here, we investigate the buckling behavior of soft, thin-shell domes with free boundary conditions to facilitate a novel gripper design. Tuning the buckling mode and pressure could enhance the gripper's actuation speed and energy input characteristics. We will present analytical and numerical models used to find the dome's buckling pressure as a function of its material and geometric properties, such as its slenderness ratio and spherical cap angle. We analyze the shell's quasi-static response during pressurization with finite element methods (FEM), and we discuss good agreement between the two models. For experimental validation, we investigate a hemispherical soft shell with a thin film covering the leading edge to create an isolated fluid cavity. Using similar FEM, we model the fluid cavity to compare to the experimental results. Our research aims to advance knowledge about the buckling of soft, thin-shell domes and to inform the usage of hemispherical shells as soft grippers. This investigation also has the potential to pave a path forward for future research into soft devices with buckling controlled actuation.

American Physical Society March Meeting 2023

Transition Waves in Multistable Mechanical Metamaterials under Periodic Excitations

Michael Bonthron, Eleonora Tubaldi

Multistable mechanical metamaterials have recently provided unique platforms with wave guidance and wave steering capabilities. Arrays of symmetric/asymmetric bistable elements, such as elastically/plastically deformed arches, can be patterned to control the signal transmission and to passively achieve nonreciprocal wave propagation. In this work, sequential nonlinear transition wavefronts will be obtained by imposing an oscillatory transverse displacement to an element of a multistable metamaterial made of one-dimensional array of arches. Different excitation amplitude and frequencies will be considered and it will be showed how these features can be harnessed to control transition wavefronts propagation. In addition, the effect of multiple oscillatory inputs at different positions of the array will be investigated. The dynamics of the system will be theoretically studied by solving the continuum equations of motion with both direct time integration methods and numerical continuation techniques (i.e., pseudo-arclength continuation). Nonlinear wave interference, collision-like cases, and internal resonance phenomena will be characterized based on different oscillatory excitation parameters. This study will open avenues to the use of dynamic inputs to control sequential transition wavefronts in mechanical metamaterials.

American Physical Society March Meeting 2023

Bioinspired Soft Metamaterials for Soft Robotic Applications

Anthony Jones, Eleonora Tubaldi

In the field of soft robotics, the nonlinearities and large deformations typical of elastomeric materials are used to emulate complex phenomena typical of the animal kingdom. Introducing metamaterials further expands the design space of soft robotics, unlocking interesting and useful mechanical properties and granting novel functionalities to components. Here, we study a soft metamaterial that undergoes a global pattern reconfiguration due to buckling instability upon inflation. Studies thus far have induced global buckling instability through uniaxial compression or negative pressure, but we will introduce and leverage, for the first time, a global buckling instability due to positive pressure in a porous metamaterial. Both numerical and experimental approaches will be used to obtain pressure-volume curves and to characterize the buckling modes based on the geometrical properties of the 3D porous metamaterial. We will show how the buckling-induced reconfigurable and reversible behavior can be harnessed for gripping and sequential actuation purposes. Leveraging this instability as a means for actuation, we explore a new class of programmable soft actuators with distributed gripping and embedded sequencing capabilities.

American Physical Society March Meeting 2023

2023

2022 MRC-ARL Summer Student Team Research Experience: Air-Deployed Robots for Mobile Sensing

S. Gart, M. Bundy, R. Vonwahlde, K. Jayarajah, K. Rippy, F. Ayoub, J. Chen, H. Zhuang, Z. Nzeako, S. Muntaha, K. Nwaiwu, W. Mackinnon, S. Mukku, D. Aderinwale, B. Lin, J. Scheffler, D. Zhu, M. Zheng, S. Roy, J. Mirenzi, B. Amaya, D. Hu, Huan Xu, Derek Paley

The MRC-ARL Summer Student Team Research Experience, a joint effort of the Maryland Robotics Center (MRC) and the US Army Combat Capabilities Development Command (DEVCOM) Army Research Laboratory (ARL), engaged 20 undergraduate students from the University of Maryland, Baltimore County and the University of Maryland, College Park. Their mission centered on the conception and construction of a system for the airdrop deployment of a ground-based robot. This was motivated by the quest for safer, more efficient alternatives to current methods of delivering critical supplies and personnel in high-risk scenarios, reducing both danger and resource intensiveness. Over the course of an 8-week program, the students focused on creating an impact energy mitigation system and developing algorithms to facilitate ground navigation post-impact. Their collective efforts culminated in a final, real-world test scenario at ARLs Robotics Research Collaboration Campus. Here, robots were dropped from a 12-m building. Although the students demonstrated solutions were only partially successful, they learned valuable lessons on teamwork and project management and set the stage for future advances in air-dropped robotic systems.

Defense Technical Information Center

Refining autonomous vehicle situational awareness due to varying sensor error

Donald Costello, Nicholas Hanlon, Huan Xu

Pilots use their senses and training to generate situational awareness (SA). They then use this SA to make sound aeronautical decisions. Autonomous vehicles, by contrast, cannot rely on pilot expertise in off-nominal situations. They must rely on their onboard sensors to build SA of the environment. As these sensors degrade, it is hypothesized that a point exists where the SA generated by these sensors is inadequate to allow the autonomous vehicle to make sound aeronautical decisions. In previous work, a point was defined based on broad assumptions within a modeling and simulation environment (i.e., the error within each sensor was known and not random). This research used a larger data set that contained random errors within the sensors. The data was then used to build predictive equations through a Monte Carlo simulation in the same simulation environment as previous work. While the data showed there was a statistically significant relationship between the error values in each sensor and the fused distance value, the resulting predictive equations were not able to provide adequate SA to make sound aeronautical decisions. This research highlights multiple issues the test and evaluation community will face when trying to develop new techniques for the verification and validation of autonomous systems.

Systems Engineering (INCOSE)

Generating certification evidence for the certification of collision avoidance in autonomous surface vessels

Donald Costello, Huan Xu

The United States Navy plans on fielding autonomous surface vessels in the near future. This paper presents a preliminary approach for certifying an autonomous surface vessel to complete a transoceanic voyage. It decomposes the rules currently outlined by the Convention on the International Regulations for Preventing Collisions at Sea for avoiding a collision when two vessels are in sight of one another in international waters.

Maritime Policy & Management

Distributed Task Allocation Algorithms for Multi-Agent Systems With Very Low Communication

Akshay Bapat, Bharath Reddy Bora, Jeffrey Herrmann, Shapour Azarm, Huan Xu, Michael Otte

Explores the problem of task allocation among networked multi-robot systems when communication conditions are poor. The authors develop solutions that ensure all tasks are eventually completed—even if some agents in the network are destroyed. Their two new task allocation algorithms assume communication may not happen, but benefit the robotic agents and their missions whenever communication is successful.

IEEE Access

Using a run time assurance approach for certifying autonomy within naval aviation

Donald Costello, Huan Xu

The methods and procedures within United States naval aviation to certify an aircraft safe for flight are well established. However, these methods and procedures are based on clearing a system that is operated or monitored by a human. A fully autonomous system will not have a human in or on the loop and will therefore require a new method for certifying it safe for flight. This paper details how to use run time assurance as the framework for a safety of flight certification of autonomous behavior within United States naval aviation.

Systems Engineering, an INCOSE journal

2022

A Path Dependent Approach for Characterizing the Legal Governance of Autonomous Systems

Huan Mumu Xu, Joseph Borson

Autonomous systems hold promise for significant improvements in fields from aircraft to cars to medical devices. These systems will be subject to legal governance requirements. While some regulatory principles apply in general to autonomous systems, an ‘‘autonomy-focused’’ approach is an inadequate lens to consider their governance. A better approach takes into consideration that autonomous systems are improvements of currently regulated complex systems, and that the regulation of autonomous elements will occur within those systems’ preexisting regulatory frameworks. In this work, the authors describe what such a framework might look like.

IEEE Access

2021

Multi-Agent Ergodic Coverage in Urban Environments

Shivang Patel, Senthil Hariharan, Pranav Dhulipala, Ming C Lin1, Dinesh Manocha, Huan Xu, Michael Otte

An important aspect of dynamic urban coverage is how building collision avoidance is incorporated into the overall coverage mission. We consider a multi-agent urban dynamic coverage problem in which a team of flying agents uses downward facing cameras to observe the street-level environment outside of buildings.

GAMMA Lab website; presented at ICRA 2021

2020

Autonomous Flight-Test Data in Support of Safety of Flight Certification

Donald H. Costello III, Jason Jewell, Huan Xu

The current safety of flight clearances for unmanned aircraft requires a qualified operator who can make decisions and ultimately bears the responsibility for the safe operations of the vehicle. The future of aviation is unmanned, and ultimately autonomous. Yet, a method for certifying an autonomous vehicle to make decisions currently reserved for qualified pilots does not exist. Before we can field autonomous systems, a process needs to be approved to certify them.

This paper analyzes the flight-test data (both developmental and operational) of an autonomous decision engine selecting an appropriate landing site for a large rotorcraft in an unprepared landing zone. In particular, this paper focuses on using legacy test and evaluation methods to determine their suitability for obtaining a safety of flight clearance for a system that possesses autonomous functionality.

The autonomous system under test was able to complete a mission currently reserved for qualified pilots under controlled conditions. However, when confronted with conditions that were not anticipated (or programmed), the software lacked the judgment a pilot uses to complete a mission under off-nominal conditions.

AIAA Journal of Air Transportation

Mitigation of Ground Impact Hazard for Safe Unmanned Aerial Vehicle Operations

Andrew Poissant, Lina Castano, Huan Xu

The researchers have developed a UAV hazard mitigation software that guards against ground impact in highly populated areas. Successful incorporation of the ground impact and hazard mitigation (GIHM) module into UAV software reduces fatalities per flight hour and brings UAVs closer to being safe enough for integration into the NAS.

AIAA Journal of Aerospace Information Systems

Multi-Agent Coverage in Urban Environments

Shivang Patel, Senthil Hariharan, Pranav Dhulipala, Ming C Lin, Dinesh Manocha, Huan Xu, Michael Otte

A study of multi-agent coverage algorithms for autonomous monitoring and patrol in urban environments. The researchers consider scenarios in which a team of flying agents uses downward facing cameras (or similar sensors) to observe the environment outside of buildings at street-level. They conduct an empirical evaluation of six multi-agent coverage algorithms in urban environments.

arXiv.org

Decentralized Task Allocation in Multi-Agent Systems Using a Decentralized Genetic Algorithm

Ruchir Patel, Eliot Rudnick-Cohen, Shapour Azarm, Michael Otte, Huan Xu, Jeffrey W. Herrmann

In multi-agent collaborative search missions, task allocation is required to determine which agents will perform which tasks. The paper proposes a new approach for decentralized task allocation based on a decentralized genetic algorithm.

2020 International Conference on Robotics and Automation

Experimental Comparison of Decentralized Task Allocation Algorithms Under Imperfect Communication

Sharan Nayak, Mohamed Khalid M. Jaffar, Estefany Carrillo, Suyash Yeotikar, Eliot Rudnick-Cohen, Ruchir Patel, Jeffrey Herrmann, Huan Xu, Shapour Azarm, Michael Otte

An experimental comparison of the performance of five state-of-the-art decentralized task allocation algorithms under imperfect communication conditions for teams of unmanned aerial vehicles (UAVs).

IEEE Robotics and Automation Letters

2022

UIVNAV: Underwater Information-driven Vision-based Navigation via Imitation Learning

Xiaomin Lin, Nare Karapetyan, Kaustubh Joshi, Tianchen Liu, Nikhil Chopra, Miao Yu, Pratap Tokekar, Yiannis Aloimonos

Autonomous navigation in the underwater environment is challenging due to limited visibility, dynamic changes, and the lack of a cost-efficient accurate localization system. We introduce UIVNAV, a novel end-to-end underwater navigation solution designed to drive robots over Objects of Interest (OOI) while avoiding obstacles, without relying on localization. UIVNAV uses imitation learning and is inspired by the navigation strategies used by human divers who do not rely on localization.

arXiv.org

Cardynet: Deep Learning-based Navigation for Car-like Robots in Dynamic Environments

Gurtajbir Singh Herr, Lasitha Weerakoon, Miao Yu, Nikhil Chopra

The authors utilize an optimization based planner, namely TEB local planner, as the expert to generate high quality motion commands for a car-like robot operating in a simulated dynamic environment. They use these labels to train a deep neural network that learns to navigate. The deep learning-based planner is further augmented with safety margins to enhance its effectiveness in collision avoidance.

Proceedings of the ASME 2022 International Mechanical Engineering Congress and Exposition IMECE2022

Learning-based Autonomous Underwater Vehicle Navigation Following Human Actions in Confined Environment

Tianchen Liu, Miao Yu, Nikhil Chopra

Navigation of autonomous underwater vehicles (AUVs) is critical for many vital tasks operated in the aquatic environment. However, the navigation problem is challenging in confined environments like caves. Since humans can usually solve such a problem with minimal effort, the main objective of this paper is to implement learning-based neural network models for AUVs to learn from human actions in a confined environment. The Girona underwater dataset, acquired in a cave exploration experiment, was used for the learning task. The odometry and sonar data from the dataset were used to train two deep neural network models for AUV navigation through the underwater caves. The test data demonstrates that both models achieve relatively high accuracy for the navigation task.

IEEE OCEANS 2022 Conference, Hampton Roads, Va.

Cartographer glass: 2D Graph SLAM Framework using LiDAR for Glass Environments

Lasitha Weerakoon, Gurtajbir Singh Herr, Jasmine Blunt, Miao Yu, Nikhil Chopra

The authors study algorithms for detecting and including glass objects in an optimization-based Simultaneous Localization and Mapping (SLAM) algorithm for safe and reliable robot navigation. They utilize a simple and computationally inexpensive glass detection scheme for detecting glass objects and present the methodology to incorporate the identified objects into the occupancy grid maintained by such an algorithm (Google Cartographer).

arXiv.org

SeaDroneSim: Simulation of Aerial Images for Detection of Objects Above Water

Xiaomin Lin, Cheng Liu, Allen Pattillo, Miao Yu, Yiannis Aloimonos

SeaDroneSim is a new benchmark suite that can be used to create photo-realistic aerial image datasets with ground truth for segmentation masks of any given object.

Computer Vision Foundation 2023 WACV Workshop

2021

Detecting and Counting Oysters

Behzad Sadrfaridpour, Yiannis Aloimonos, Miao Yu, Yang Tao, Donald Webster

To test the idea that advancements in robotics and artificial intelligence offer the potential to improve the monitoring of oyster beds, the researchers prepared a remote operated underwater vehicle (ROV) with a camera and filmed in the Chesapeake Bay. They then used these videos to train convolutional neural networks (CNNs) to count oysters and track them in consecutive image frames so they are not identified multiple times.

arXiv.org

2020

Multi-Agent Coverage in Urban Environments

Keshav Rajasekaran, Hyung Dae Bae, Sarah Bergbreiter, Miao Yu

A novel method to fabricate a micro-scale artificial hair sensor capable of directional airflow sensing.

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

About the Maryland Robotics Center

Learn about the Maryland Robotics Center's research and education opportunities in this video.

Read More

Top