Bio-inspired robots; medical robots; collaborative, cooperative and networked robots; robotics for extreme environments; unmanned air and water vehicles; AI and computer vision for robots

ISR faculty have advanced flocking and swarming theory, control and design for robotic groups and swarms. ISR developed motion description languages; designed and fabricated a modular dexterous hand; made advances in underwater robots, flapping-wing micro air vehicles and micro robots; and introduced innovations in computer vision and hyperdimensional computing theory for robots. A multi-part, in-mold assembly process that reduces process cost and enables robotic design possibilities created articulated structures without requiring post-molding assembly steps. ISR started the Maryland Robotics Center as one of its initiatives in 2010.

Recent publications


Preparing for Geophysical Science Enabled by Crewed and Robotic Missions on the Surface of the Moon

Schmerr, N.; Richardson, J.; Ghent, R.; Siegler, M.; Young, K.; Wasser, M.; Whelley, P.; Buczkowski, D.; Carter, L.; Connor, C.; Connor, L.; Bleacher, J.; Fouch, M.; Baker, D.; Hurford, T.; Jozwiak, L.; Kruse, S.; Lekic, V.; Naids, A.; Porter, R. Montesi, L.; Richardson, D. C.; Rumpf, E.; Schorghofer, N.; Sunshine, J.; Goossens, S.; Whelley, N.; Wyrick, D.; Zhu, W.; Bell, E.; DeMartini, J.; Coan, D.; Akin, D.; Cohen, B.; Mazarico, E.; Neal, C.; Panning, M.; Petro, N.; Strauss, B.; Weber, R.; Glotch, T.; Hendrix, A.; Parker, A.; Wright, S.

Geophysics on the Moon will be an important tool for identifying key targets for geological prospecting, scientific sampling, ISRU, assessing hazards and risks to crews and infrastructure, and determining the deep workings of the lunar interior.

Lunar Surface Science Workshop, held virtually May 28-29, 2020


PRGFlow: Benchmarking SWAP-Aware Unified Deep Visual Inertial Odometry

Nitin J. Sanket, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos

The authors present a simple way to estimate ego-motion/odometry on an aerial robot using deep learning combining commonly found on-board sensors: a up/down-facing camera, an altimeter source and an IMU.


EVDodgeNet: Deep Dynamic Obstacle Dodging with Event Cameras

Nitin J. Sanket, Chethan M. Parameshwara, Chahat Deep Singh, Ashwin V. Kuruttukulam, Cornelia Fermüller, Davide Scaramuzza, Yiannis Aloimonos

A deep learning-based solution for dodging multiple dynamic obstacles on a quadrotor with a single event camera and on-board computation.

2020 IEEE International Conference on Robotics and Automation (ICRA)



Bilateral Teleoperation of Soft Robots under Piecewise Constant Curvature Hypothesis: An Experimental Investigation

Lasitha Weerakoon, Nikhil Chopra

This paper investigates an adaptive task space bilateral teleoperation framework for soft robots with dynamic uncertainties. It assumes a non-redundant rigid master manipulator and a redundant soft slave manipulator under the piecewise constant curvature hypothesis.

2020 American Control Conference


Robotic Parasitic Array Optimization in Outdoor Environments

Jeffrey Twigg, Fikadu Dagefu, Nikhil Chopra, Brian M. Sadler

The paper describes a two-element parasitic array formed by two ground robots and proposes a technique by which this array can optimize its gain in a direction of interest online.

2019 IEEE International Symposium on Safety, Security, and Rescue Robotics


Decentralized Task Allocation in Multi-Agent Systems Using a Decentralized Genetic Algorithm

Ruchir Patel, Eliot Rudnick-Cohen, Shapour Azarm, Michael Otte, Huan Xu, Jeffrey W. Herrmann

In multi-agent collaborative search missions, task allocation is required to determine which agents will perform which tasks. The paper proposes a new approach for decentralized task allocation based on a decentralized genetic algorithm.

2020 International Conference on Robotics and Automation

Experimental Comparison of Decentralized Task Allocation Algorithms Under Imperfect Communication

Sharan Nayak, Mohamed Khalid M. Jaffar, Estefany Carrillo, Suyash Yeotikar, Eliot Rudnick-Cohen, Ruchir Patel, Jeffrey Herrmann, Huan Xu, Shapour Azarm, Michael Otte

An experimental comparison of the performance of five state-of-the-art decentralized task allocation algorithms under imperfect communication conditions for teams of unmanned aerial vehicles (UAVs).

IEEE Robotics and Automation Letters

Data-driven Metareasoning for Collaborative Autonomous Systems

Jeffrey Herrmann

A novel data-driven metareasoning approach that generates a metareasoning policy that agents in a multi-agent system can use whenever they must collaborate to assign tasks.









ISR Technical Report, DRUM


COVID-Robot: Monitoring Social Distancing Constraints in Crowded Scenarios

Adarsh Jagan Sathyamoorthy, Utsav Patel, Yash Ajay Savle, Moumita Paul, Dinesh Manocha

Maintaining social distancing norms between humans has become an indispensable precaution to slow dowthe transmission of COVID-19. The paper presents a novel method to automatically detect pairs of humans in a crowded scenario who are not adhering to the 6-foot social distance constraint. The approach makes no assumption about crowd density or pedestrian walking directions. A mobile robot is fitted with commodity sensors, namely an RGB-D camera and a 2-D lidar to perform collision-free navigation in a crowd and estimate the distance between all detected individuals in the camera’s field of view. The robot is also equipped with a thermal camera that wirelessly transmits thermal images to security/healthcare personnel who monitor if any individual exhibits a higher than normal temperature. In indoor scenarios, the mobile robot also can be combined with static mounted CCTV cameras to further improve the performance of the number of social distancing breaches detected. The robot can accurately pursue walking pedestrians etc.

Autonomous Social Distancing in Urban Environments using a Quadruped Robot

Tingxiang Fan, Zhiming Che∗, Xuan Zhao, Jing Liang, Cong Shen, Dinesh Manocha, Jia Pan1, Wei Zhang

The paper presents a fully autonomous surveillance robot based on a quadruped platform that can promote social distancing in complex urban environments. The researchers mount multiple cameras and a 3D LiDAR on the legged robot. The robot then uses an onboard real-time social distancing detection system to track nearby pedestrian groups. Next, the robot uses a crowd-aware navigation algorithm to move freely in highly dynamic scenarios. The robot finally uses a crowd-aware routing algorithm to effectively promote social distancing by using human-friendly verbal cues to send suggestions to over-crowded pedestrians.

Accurate High Fidelity Simulations for Training Robot Navigation Policies for Dense Crowds using Deep Reinforcement Learning

Jin Liang, Ustav Patel, Adarsh Sathyamoorthy, Dinesh Manocha

A novel high fidelity 3-D simulator that significantly reduces the sim-to-real gap in mobile robotics training collision avoidance policies based on Deep Reinforcement Learning for dense crowd scenarios.

AutoTrajectory: Label-free Trajectory Extraction and Prediction from Videos using Dynamic Points

Yuexin Ma, Xinge Zhu, Xinjing Cheng, Ruigang Yang, Jiming Liu, Dinesh Manocha

For intelligent agents like robots and autonomous vehicles, it is crucial to be able to forecast neighboring traffic-agents’ future trajectories for navigation and planning applications. Current methods for trajectory prediction operate in super-vised manners, and therefore require vast quantities of corresponding ground truth data for training. The paper presents a novel, label-free algorithm, AutoTrajectory, for trajectory extraction and prediction to use raw videos directly.

DenseCAvoid: Real-time Navigation in Dense Crowds using Anticipatory Behaviors

Adarsh Jagan Sathyamoorthy, Jing Liang, Utsav Patel, Tianrui Guan, Rohan Chandra, Dinesh Manocha

A navigation algorithm for navigating a robot through dense crowds and avoiding collisions by anticipating pedestrian behaviors. The formulation uses visual sensors and a pedestrian trajectory prediction algorithm to track pedestrians in a set of input frames and provide bounding boxes that extrapolate the pedestrian positions in a future time. This hybrid approach combines trajectory prediction with a Deep Reinforcement Learning-based collision avoidance method to train a policy to generate smoother,safer, and more robust trajectories during run-time.

Efficient Multi-Agent Motion Planning in Continuous Workspaces Using Medial-Axis-Based Swap Graphs

Liang He, Zherong Pan, Biao Jia, Dinesh Manocha

An algorithm for homogeneous, labeled, and disk-shaped multi-agent motion planning in continuous workspaces with arbitrarily-shaped obstacles.

ProxEmo: Gait-based Emotion Learning and Multi-view Proxemic Fusion for Socially-Aware Robot Navigation

Venkatraman Narayanan, Bala Murali Manoghar, Vishnu Sashank Dorbala, Dinesh Manocha, Aniket Bera

ProxEmo is an end-to-end emotion prediction algorithm for socially aware robot navigation among pedestrians. This approach predicts the perceived emotions of a pedestrian from walking gaits, which is then used for emotion-guided navigation taking into account social and proxemic constraints. To classify emotions, ProxEmo uses a multi-view skeletongraph convolution-based model that works on a commodity camera mounted onto a moving robot. The emotion recognition is integrated into a mapless navigation scheme and makes no assumptions about the environment of pedestrian motion.

Frozone: Freezing-Free, Pedestrian-Friendly Navigation in Human Crowds

Adarsh Jagan Sathyamoorthy, Utsav Patel, Tianrui Guan, Dinesh Manocha

Frozone is a novel algorithm to deal with the Freezing Robot Problem that arises when a robot navigates through dense scenarios and crowds. The method senses and explicitly predicts the trajectories of pedestrians and constructs a Potential Freezing Zone (PFZ); a spatial zone where the robot could freeze or be obtrusive to humans. The formulation computes a deviation velocity to avoid the PFZ, which also accounts for social constraints. Furthermore, Frozone is designed for robots equipped with sensors with a limited sensing range and field of view. The researchers ensure that the robot’s deviation is bounded, thus avoiding sudden angular motion which could lead to the loss of perception data of the surrounding obstacles.

Learning Resilient Behaviors for Navigation Under Uncertainty

Tingxiang Fan, Pinxin Long, Wenxi Liu, Jia Pan, Ruigang Yang, Dinesh Manocha

An uncertainty-aware navigation approach for mobile robots that introduces a predictor to model the environmental uncertainty. In addition, the paper proposes an uncertainty-aware navigation network to learn resilient behaviors in prior unknown environments.

2020 International Conference on Robotics and Automation

CrowdSteer: Realtime Collision Avoidance for Mobile Robots in Dense Crowds using Implicit Multi-sensor Fusion and Deep Reinforcement Learning

Jing Liang, Utsav Patel, Adarsh Jagan Sathyamoorthy, Dinesh Manocha

CrowdSteer is a learning-based collision avoidance algorithm for mobile robots operating in dense and crowded environments.

19th International Conference on Autonomous Agents and Multiagent Systems

OF-VO: Reliable Navigation among Pedestrians Using Commodity Sensors

Jing Liang, Yi-Ling Qiao, Dinesh Manocha

An algorithm that uses commodity visual sensors, including RGB-D cameras and a2D lidar for safe navigation of a mobile robot among pedestrians. The algorithm explicitly predicts the velocities and positions of surrounding obstacles through optical flow estimation and object detection.

3D-OGSE: Online Smooth Trajectory Generation for Quadrotors using Generalized Shape Expansion in Unknown 3D Environments

Vrushabh Zinage, Senthil Hariharan Arul, Dinesh Manocha

An online motion planning algorithm for generating smooth, collision-free trajectories for quadrotors operating in an unknown, cluttered 3D environment.

How are You Feeling? Multimodal Emotion Learning for Socially-Assistive Robot Navigation

Aniket Bera, Tanmay Randhavane, Rohan Prinja, Kyra Kapsaskis, Austin Wang, Kurt Gray, Dinesh Manocha

A real-time algorithm that learns the emotion state of the pedestrians to perform socially-aware navigation. The robot learns pedestrians’ emotions and their proxemic constraints to improve both social comfort and navigation.

15th IEEE International Conference on Automatic Face and Gesture Recognition

HMPO: Human Motion Prediction in Occluded Environments for Safe Motion Planning

Jae Sung Park, Dinesh Manocha

A new approach to generate collision-free trajectories for a robot operating in close proximity with a human obstacle in an occluded environment.

Deep Differentiable Grasp Planner for High-DOF Grippers

Min Liu, Zherong Pan, Kai Xu, Kanishka Ganguly, Dinesh Manocha

An end-to-end algorithm for training deep neural networks to grasp novel objects.

Grasping Fragile Objects using a Stress-Minimization Metric

Zherong Pan, Xifeng Gao, Dinesh Manocha

A new stress-minimization metric to generate optimal grasps for brittle and fragile objects.


Reinforcement Learning-Based Visual Navigation with Information-Theoretic Regularization

Qiaoyun Wu, Kai Xu, Jun Wang, Mingliang Xu, Dinesh Manocha

The authors integrate an information-theoretic regularization into a deep reinforcement learning framework for the target-driven task of visual navigation in robotics. This is achieved by first learning to generate a next observation from a current observation and a navigation target, then planning an action toward the target based on the generated observation and the current observation.

New Formulation of Mixed-Integer Conic Programming for Globally Optimal Grasp Planning

Min Liu, Zherong Pan, Kai Xu, and Dinesh Manocha

Classical robotics research presents the first grasp planning algorithm to compute globally optimal gripper pose that maximizes a grasp metric.

DCAD: Decentralized Collision Avoidance with Dynamics Constraints for Agile Quadrotor Swarms

Senthil Hariharan Arul, Dinesh Manocha

A new algorithm for decentralized collision avoidance for quadrotor swarm navigation in dense environments with static and dynamic obstacles.

3-D Deformable Object Manipulation using Deep Neural Networks

Zhe Hu, Tao Han, Peigen Sun, Jia Pan, Dinesh Manocha

Deep neural network-based controller to servo-control position and shape of deformable objects with unknown deformation properties.

IEEE Robotics and Automation Letters

DeepMNavigate: Deep Reinforced Multi-Robot Navigation Unifying Local & Global Collision Avoidance

Qingyang Tan, Tingxiang Fan, Jia Pan, Dinesh Manocha

A novel algorithm (DeepMNavigate) for global multi-agent navigation in dense scenarios using deep reinforcement learning. | Watch a video about DeepMNavigate |

GraphRQI: Classifying Driver Behaviors Using Graph Spectrums

Rohan Chandra, Uttaran Bhattacharya, Trisha Mittal, Xiaoyu Li, Aniket Bera, Dinesh Manocha

The GraphRQI algorithm identifies driver behaviors from road agent trajectories. It is 25 percent more accurate over prior behavior classification algorithms for autonomous vehicles.  | Watch a video about GraphRQI |

Realtime Simulation of Thin-Shell Deformable Materials using CNN-Based Mesh Embedding

Qingyang Tan, Zherong Pan, Lin Gao, and Dinesh Manocha

A new method bridges the gap between mesh embedding and physical simulation for efficient dynamic models of clothes. The key technique is a graph-based convolutional neural network (CNN) defined on meshes with arbitrary topologies, and a new mesh embedding approach based on physics-inspired loss term. After training, the learned simulator runs10–100 times faster and the accuracy is high enough for robot manipulation tasks. | Watch a video about this method |


Tracking Performance of Model-Based Thruster Control of a Remotely Operated Underwater Vehicle

Jordan Boehm, Eric Berkenpas, Charles Shepard, Derek A. Paley

The paper compares output feedback control strategies for an underwater thruster.

IEEE Journal of Oceanic Engineering

Bioinspired pursuit with a swimming robot using feedback control of an internal rotor

Brian Free, Jinseong Lee, Derek Paley

The paper presents a state-feedback control law for steering a fish-inspired robot in a desired direction, and engaging it in pure pursuit.

Bioinspiration and Biomimetics

Cooperative Mapping and Target Search over an Unknown Occupancy Graph Using Mutual Information

Artur Wolek, Sheng Cheng, Debdipta Goswami, Derek Paley

A cooperative mapping and target-search algorithm for a team of autonomous quadrotors equipped with noisy, range-limited sensors. The algorithm can concurrently map and search an unknown urban area, while detecting and tracking a mobile ground target.

IEEE Robotics and Automation Letters

Optimal control of a 1D diffusion process with a team of mobile actuators under jointly optimal guidance

Sheng Cheng, Derek Paley

An optimization framework to control a distributed parameter system (DPS) using a team of mobile actuators. The optimization simultaneously seeks efficient guidance of the mobile actuators and effective control of the DPS such that an integrated cost function associated with both the mobile actuators and the DPS is minimized.

CDCL paper

Feedback Control of a Soft Swinging Appendage

Travis Burch, John Lathrop, William Scott, Derek Paley

A state-space description using planar discrete elastic rod theory of a soft robotic appendage with torque input at one end. The authors design a linear output feedback controller to balance the appendage in an unstable vertical configuration with an angle sensor and torque input co-located at the base.

CDCL paper

Geometric Attitude and Position Control of a Quadrotor in Wind

William Craig, Derrick Yeo, Derek Paley

The researchers use  a model of the aerodynamic interaction between the propellers and wind, paired with onboard flow sensing and feedback control, to improve the stability of quadrotors in unsteady winds with the long-term goal of enabling reliable outdoor flight in windy conditions.

AIAA Journal of Guidance, Control, and Dynamics

Orbit Design for Cislunar Space Domain Awareness

Erin Fowler, Stella Hurtt, Derek Paley

Quantitative assessments of the orbits and sensor characteristics of satellites intended for cislunar space domain awareness.

2nd IAA Conference on Space Situational Awareness



Output Feedback Control for Lift Maximization of a Pitching Airfoil

Justin M. Lidard, Debdipta Goswami, David Snyder, Girguis Sedky, Anya Jones, Derek Paley

Unsteady aerodynamics is driving research at the interface of fluid dynamics and control theory for low Reynolds number aircraft such as micro air vehicles (MAVs). The regulation and control of unsteady behavior is crucial for maintaining the stability of an MAV, which necessitates accurate modeling of their flight surfaces. This paper describes the implementation of the Goman-Khrabrov model for flow stagnation near an actuated airfoil with a feedback-controlled pitch rate for the purpose of maximizing the time-averaged unsteady lift.

AIAA SciTech 2020 Forum

Non-Gaussian Estimation of a Potential Flow by an Actuated Lagrangian Sensor Steered to Separating Boundaries by Augmented Observability

Francis D. Lagor, Kayo Ide, Derek Paley

An architecture for estimation of a flow field using a hypothetical oceanographic vehicle that is guided along paths of high flow-field observability, a concept quantifying the informativeness of a path.

IEEE Journal of Oceanic Engineering

Mobile Sensor Networks Control: Adaptive Sampling of Spatiotemporal Processes

Derek Paley, Artur Wolek

A review of control of mobile sensor networks for environmental monitoring and other applications. Includes sensor platform dynamics and cooperative control and estimation; process modeling and estimation for both continuous and discrete models; sampling metrics and optimization, including coverage-, topology-, information-, and estimation-based metrics; and methods for task design and allocation.

Annual Review of Control, Robotics, and Autonomous Systems

Stabilization of a Quadrotor in Wind with Flow Sensing: Linear Modeling and Control for Attitude and Position Hold

William Craig, J. T. Lewis, Derek Paley

This paper describes a linear controller that has been optimized for gust rejection using CONDUITR©, based on system identification performed with CIFERR©. Additionally, a custom flow probe package was used to investigate the benefits of flow feedback for gust rejection. Experiments were performed with a 210mm quadrotor system running Cleanflight software, where flow feedback yielded improvements for both short and long time-scale gusts, particularly for the longer time-scale five second gusts when the vehicle had time to settle in the wind.

VFS Autonomous VTOL Technical Meeting and Electric VTOL Symposium



Recognizing Hemiparetic Ankle Deficits Using Wearable Pressure Sensors

Ahmed Ramadan, Anindo Roy, Elisabeth Smela

This paper provides proof of concept for a novel method to recognize impaired push-off and foot-drop deficits in hemiparetic gait using analog pressure sensors. These data may enhance feedback from a modular ankle exoskeleton (such as Anklebot) for stroke rehabilitation.

IEEE Journal of Translational Engineering in Health and Medicine


Multi-Agent Coverage in Urban Environments

Shivang Patel, Senthil Hariharan, Pranav Dhulipala, Ming C Lin, Dinesh Manocha, Huan Xu, Michael Otte

A study of multi-agent coverage algorithms for autonomous monitoring and patrol in urban environments. The researchers consider scenarios in which a team of flying agents uses downward facing cameras (or similar sensors) to observe the environment outside of buildings at street-level. They conduct an empirical evaluation of six multi-agent coverage algorithms in urban environments.

Decentralized Task Allocation in Multi-Agent Systems Using a Decentralized Genetic Algorithm

Ruchir Patel, Eliot Rudnick-Cohen, Shapour Azarm, Michael Otte, Huan Xu, Jeffrey W. Herrmann

In multi-agent collaborative search missions, task allocation is required to determine which agents will perform which tasks. The paper proposes a new approach for decentralized task allocation based on a decentralized genetic algorithm.

2020 International Conference on Robotics and Automation

Experimental Comparison of Decentralized Task Allocation Algorithms Under Imperfect Communication

Sharan Nayak, Mohamed Khalid M. Jaffar, Estefany Carrillo, Suyash Yeotikar, Eliot Rudnick-Cohen, Ruchir Patel, Jeffrey Herrmann, Huan Xu, Shapour Azarm, Michael Otte

An experimental comparison of the performance of five state-of-the-art decentralized task allocation algorithms under imperfect communication conditions for teams of unmanned aerial vehicles (UAVs).

IEEE Robotics and Automation Letters

About the Maryland Robotics Center

Learn about the Maryland Robotics Center's research and education opportunities in this video.

Read More

ISR and Maryland Robotics Center robotics news