2020
Alexander Castro, Leo Peckerar, Timothy Horiuchi, Pamela Abshire
A nano-quadcopter system that performs odor detection in a windless indoor environment. The system uses the Crazyflie platform and incorporates a single commercially available metal oxide semiconductor gas sensor. The authors demonstrate that rotor-induced airflow enhances odor detection. It can be considered to be the quadrotor equivalent of sniffing behavior.
2020 IEEE SENSORS Conference
2020
Schmerr, N.; Richardson, J.; Ghent, R.; Siegler, M.; Young, K.; Wasser, M.; Whelley, P.; Buczkowski, D.; Carter, L.; Connor, C.; Connor, L.; Bleacher, J.; Fouch, M.; Baker, D.; Hurford, T.; Jozwiak, L.; Kruse, S.; Lekic, V.; Naids, A.; Porter, R. Montesi, L.; Richardson, D. C.; Rumpf, E.; Schorghofer, N.; Sunshine, J.; Goossens, S.; Whelley, N.; Wyrick, D.; Zhu, W.; Bell, E.; DeMartini, J.; Coan, D.; Akin, D.; Cohen, B.; Mazarico, E.; Neal, C.; Panning, M.; Petro, N.; Strauss, B.; Weber, R.; Glotch, T.; Hendrix, A.; Parker, A.; Wright, S.
Geophysics on the Moon will be an important tool for identifying key targets for geological prospecting, scientific sampling, ISRU, assessing hazards and risks to crews and infrastructure, and determining the deep workings of the lunar interior.
Lunar Surface Science Workshop, held virtually May 28-29, 2020
2023
Xiaomin Lin, Nare Karapetyan, Kaustubh Joshi, Tianchen Liu, Nikhil Chopra, Miao Yu, Pratap Tokekar, Yiannis Aloimonos
Autonomous navigation in the underwater environment is challenging due to limited visibility, dynamic changes, and the lack of a cost-efficient accurate localization system. We introduce UIVNAV, a novel end-to-end underwater navigation solution designed to drive robots over Objects of Interest (OOI) while avoiding obstacles, without relying on localization. UIVNAV uses imitation learning and is inspired by the navigation strategies used by human divers who do not rely on localization.
arXiv.org
Nitin Sanket, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos
Robots are active agents that operate in dynamic scenarios with noisy sensors. Predictions based on these noisy sensor measurements often lead to errors and can be unreliable. To this end, roboticists have used fusion methods using multiple observations. Lately, neural networks have dominated the accuracy charts for percep- tion-driven predictions for robotic decision-making and often lack uncertainty metrics associated with the pre- dictions. Here, the authors present a mathematical formulation to obtain the heteroscedastic aleatoric uncertainty of any arbitrary distribution without prior knowledge about the data. The approach has no prior assumptions about the prediction labels and is agnostic to network architecture.
Science Robotics
Snehesh Shrestha, Ge Gao, Cornelia Fermuller, Yiannis Aloimonos
About this paper, the authors write: "Many of us researchers take extra measures to control for known-unknowns. However, unknown-unknowns can, at best, be negligible, but otherwise, they could produce unreliable data that might have dire consequences in real-life downstream applications. Human-Robot Interaction standards informed by empirical data could save us time and effort and provide us with the path toward the robots of the future. To this end, we share some of our pilot studies, lessons learned, and how they affected the outcome of our experiments. While these aspects might not be publishable in themselves, we hope our work might save time and effort for other researchers towards their research and serve as additional considerations for discussion at the workshop."
github.io
Snehesh Shrestha, Yantian Zha, Ge Gao, Cornelia Fermuller, Yiannis Aloimonos
The work introduces NatSGD, a multimodal HRI dataset that contains human commands as speech and gestures, along with robot behavior in the form of synchronized demonstrated robot trajectories. These data enable HRI with Imitation Learning so that robots can learn to work with humans in challenging, real-life domains such as performing complex tasks in the kitchen.
github.io
2022
Xiaomin Lin, Cheng Liu, Allen Pattillo, Miao Yu, Yiannis Aloimonos
SeaDroneSim is a new benchmark suite that can be used to create photo-realistic aerial image datasets with ground truth for segmentation masks of any given object.
Computer Vision Foundation 2023 WACV Workshop
Yiannis Aloimonos, Giulio Sandini
Chapter 14 in the MIT Press book, Cognitive Robotics, edited by Angelo Cangelosi and Minoru Asada. This volume is part of the Intelligent Robotics and Autonomous Agents series.
In Cognitive Robotics (The MIT Press, 2022)
Xiaomin Lin, Nitin J. Sanket, Nare Karapetyan, Yiannis Aloimonos
A new way to mathematically model oysters and render images of oysters in simulation. This new method can boost detection performance with minimal real data, especially when used in conjunction with underwater robots.
arXiv.org
Levi Burner, Anton Mitrokhin, Yiannis Aloimonos, Cornelia Fermüller
A new event camera dataset, EVIMO2, is introduced that improves on the popular EVIMO dataset by providing more data, from better cameras, in more complex scenarios.
arXiv.org
2021
EVPropNET: Detecting drone by finding propellers for mid-air landing and following
Yiannis Aloimonos, Cornelia Fermüller, Nitin Sanket, Chahat Deep Singh, Chethan Parameshwara, Guido C.H.E. de Croon
The authors use the most ubiquitous part of a drone—the propeller—as the basis for a new detection scheme that is 92% accurate.
2021 Robotics Science and Systems Conference (RSS 2021)
Behzad Sadrfaridpour, Yiannis Aloimonos, Miao Yu, Yang Tao, Donald Webster
To test the idea that advancements in robotics and artificial intelligence offer the potential to improve the monitoring of oyster beds, the researchers prepared a remote operated underwater vehicle (ROV) with a camera and filmed in the Chesapeake Bay. They then used these videos to train convolutional neural networks (CNNs) to count oysters and track them in consecutive image frames so they are not identified multiple times.
arXiv.org
2020
Nitin J. Sanket, Chahat Deep Singh, Varun Asthana, Cornelia Fermüller, Yiannis Aloimonos
A framework for quadrotor navigation based on a stereo camera system whose baseline can be adapted on the fly.
arXiv.org
Chengxi Ye, Anton Mitrokhin, Cornelia Fermüller, James Yorke, Yiannis Aloimonos
Presents an unsupervised learning pipeline for dense depth, optical flow and egomotion estimation for autonomous driving applications, using the event-based output of the Dynamic Vision Sensor as input.
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Kanishka Ganguly, Behzad Sadrfaridpour, Krishna Bhavithavya Kidambi, Cornelia Fermüller, Yiannis Aloimonos
An adaptive,compliant grasping strategy for robots using only tactile feedback.
arXiv.org
Nitin J. Sanket, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos
The authors present a simple way to estimate ego-motion/odometry on an aerial robot using deep learning combining commonly found on-board sensors: a up/down-facing camera, an altimeter source and an IMU.
arXiv.org
Nitin J. Sanket, Chethan M. Parameshwara, Chahat Deep Singh, Ashwin V. Kuruttukulam, Cornelia Fermüller, Davide Scaramuzza, Yiannis Aloimonos
A deep learning-based solution for dodging multiple dynamic obstacles on a quadrotor with a single event camera and on-board computation.
2020 IEEE International Conference on Robotics and Automation (ICRA)
2023
Clinton Enwerem, John Baras
Considers the problem of safely coordinating ensembles of identical autonomous agents to conduct complex missions with conflicting safety requirements and under noisy control inputs. Using non-smooth control barrier functions (CBFs) and stochastic model-predictive control as springboards, and by adopting an extrinsic approach where the ensemble is treated as a unified dynamic entity, the authors devise a method to synthesize safety-aware control inputs for uncertain collectives, drawing upon recent developments in Boolean CBF composition and extensions of CBFs to stochastic systems.
arXiv.org
Faizan M. Tariq, David Isele, John Baras, Sangjae Bae
A risk-aware crash mitigation system (RCMS) to augment any existing motion planner (MP) that enables an autonomous vehicle to perform evasive maneuvers in high-risk situations and minimize the severity of collision if a crash is inevitable.
arXiv.org
Clinton Enwerem, John Baras
Consensus-based control laws for multiagent formation tracking in finite-dimensional state space, with the agents represented by a more general class of dynamics: control-affine nonlinear systems. The agents also exchange information via a leader-follower communication topology modeled as an undirected and connected graph with a single leader node.
arXiv.org
Clinton Enwerem, John Baras, Danilo Romero
A distributed optimal multiagent control scheme for quadrotor formation tracking under localization errors. This control architecture is based on a leader-follower approach, where a single leader quadrotor tracks a desired trajectory while the followers maintain their relative positions in a triangular formation.
arXiv.org
Nilesh Suriyarachchi, Rien Quirynen, John Baras, Stefano Di Cairano
Coordinating the flow of traffic through urban areas with multiple intersections is a complex problem whose solution has the potential to improve safety, increase throughput, and optimize energy efficiency. In addition to controlling traffic lights, the introduction of connected and automated vehicles (CAVs) offers opportunities in terms of additional sensing and actuation points within the traffic network.
This paper proposes a centralized and a decentralized implementation for the joint coordination and control of both traffic signals and mixed traffic, including CAVs and human-driven vehicles (HDVs), in a network of multiple connected traffic intersections. Mixed-integer linear programming (MILP) is used to compute safe control trajectories for both CAVs and traffic light signals, which minimize overall congestion and fuel consumption. Our approaches are validated using extensive traffic simulations on the SUMO platform and they are shown to provide improvements of around 32-60%, 90-96% and 40-60% in travel time, waiting time and fuel consumption, respectively, when compared to gap-based adaptive and timed traffic lights.
American Control Conference, 2023
Amoolya Tirumalai, Christos Mavridis, John Baras
In this work, the researchers study the inverse problem of identifying complex flocking dynamics in a domain cluttered with obstacles. Their inspiration comes from animal flocks moving in complex ways with capabilities far beyond what current robots can do. Owing to the difficulty of observing and recovering the trajectories of the agents, the authors focus on the dynamics of their probability densities, which are governed by partial differential equations (PDEs), namely compressible Euler equations subject to non-local forces. The authors formulate the inverse problem of learning interactions as a PDE-constrained optimization problem of minimizing the squared Hellinger distance between the histogram of the flock and the distribution associated to their PDEs.
arXiv.org
Amoolya Tirumalai, John Baras
Highway vehicular traffic is an inherently multi-agent problem. Traffic jams can appear and disappear mysteriously. Here, the researchers develop a method for traffic flow control that is applied at the vehicular level via mean-field games.
IEEE 61st Conference on Decision and Control
Nilesh Suriyarachchi, Erfaun Noorani, Faizan Tariq, John Baras
Proposes the use of a multi-agent shared policy reinforcement learning algorithm which leverages communication between connected autonomous vehicles (CAVs) for improved observability of downstream traffic conditions. A key feature of this method is the ability to perform shock wave dissipation control without the need for global information and the applicability of this method to multi-lane mixed traffic highways of arbitrary structure.
IEEE 61st Conference on Decision and Control
Faizan Tariq, David Isele, John Baras, Sangjae Bae
This paper proposes a hierarchical autonomous vehicle navigation architecture, composed of a high-level speed and lane advisory system (SLAS) coupled with low-level trajectory generation and trajectory following modules. Specifically, the authors target a multi-lane highway driving scenario where an autonomous ego vehicle navigates in traffic. They propose a novel receding horizon mixed-integer optimization based method for SLAS with the objective to minimize travel time while accounting for passenger comfort.
IEEE 61st Conference on Decision and Control
2022
Praveen Kumar Menaka Sekar, John Baras
Constant rise in industrial usage of robots for commercial applications has led to the need for rapid, efficient, and reliable robotic system development processes. The idea of collaborative modeling and simulation provides different views to the stakeholders, enhancing the decision-making process towards design, development, and deployment. This paper applies Model-Based Systems Engineering (MBSE) principles to a robot grasping system performing a simple pick and place task; integrates Cameo Systems Modeler (CSM), CoppeliaSim robot simulator, and Gurobi Optimizer to facilitate and accelerate the system design process; and describes a simulation-based Verification & Validation (V&V) framework and performs design space exploration. This is an effort to integrate tools to reduce the burden on system design, integration, and deployment of dexterous grasping and manipulation systems.
INCOSE International Symposium Conference on Systems Engineering Research (CSER) 2022, Norwegian University of Science and Technology
Nilesh Suriyarachchi, Faizan Tariq, Christos Mavridis, John Baras
Intersections without traffic signals are the sites of 40% of all crashes, 50% of serious collisions, and 20% of fatalities. When motorists approach these intersections, which often have “four-way” stop signs, and feature “unprotected” left turns, they must make choices. If two vehicles approach at approximately the same time, or a driver decides not to stop or wait their turn, safety is at issue. In the future, autonomous vehicles equipped with vehicle-to-infrastructure (V2I) communication may be able to provide solutions for handling these scenarios. Such vehicles, called connected autonomous vehicles (CAVs), will be able to provide multiple sensing and actuation points in the traffic flow. GAMEOPT is a novel hybrid approach to cooperative intersection control for dynamic, multi-lane, unsignalized intersections.
arXiv.org
2021
Christos Mavridis, Amoolya Tirumalai, John Baras
Considers the problem of understanding the coordinated movements of biological or artificial swarms. The authors propose a learning scheme to estimate the coordination laws of interacting agents from observations of the swarm’s density over time. Dynamics of the swarm are described based on pairwise interactions according to a Cucker-Smale flocking model, and express the swarm’s density evolution as the solution to a system of mean-field hydrodynamic equations. A new family of parametric functions is proposed to model the pairwise interactions. The augmented system is incorporated in an iterative optimization scheme to learn the dynamics of the interacting agents from observations of the swarm’s density evolution over time.
arXiv.org
Amoolya Tirumalai, Christos Mavridis, John Baras
Euler alignment systems appear as hydrodynamic limits of interacting self-propelled particle systems such as the (generalized) Cucker-Smale model. In this work, the authors study weak solutions to an Euler alignment system on smooth, bounded domains. The work falls into the general category of using insights gained from biological flocking and swarming to construct controls for large teams of UAVs.
arXiv.org
Nilesh Suriyarachchi, Faizan Tariq, Christos Mavridis, John Baras
Highway on-ramp merge junctions remain a major bottleneck in transportation networks. However, with the introduction of Connected Autonomous Vehicles (CAVs) with advanced sensing and communication capabilities modern algorithms can capitalize on the cooperation between vehicles. This paper enhances highway merging efficiency by optimally coordinating CAVs in order to maximize the flow of vehicles while satisfying all safety constraints.
2021 IEEE International Intelligent Transportation Systems Conference (ITSC)
Charles Meehan, John Baras
The authors use Model-Based Systems Engineering techniques to verify system requirements and validate stakeholder requirements for the problem of detecting and correcting for object slippage within a dexterous five-fingered robotic hand.
2021 IEEE International Systems Conference (SysCon)
John Baras, Aneesh Raghavan
Revisits the structure of events in a multi-agent system and introduces necessary new models that incorporate incompatible events in formalism. These models are essential for building non-commutative probability models.
Communications in Information and Systems
2020
Christos Mavridis, Nilesh Suriyarachchi, John Baras
Considers the problem of defending against adversarial attacks from UAV swarms performing complex maneuvers,driven by multiple, dynamically changing, leaders.
GameSec 2020 Conference
Zhenyu Lin, John Baras
an optimization-based approach for robot planning, monitoring and self-correction problems under signal temporal logic specifications (STL).
21st IFAC World Congress
Christos Mavridis, John Baras, Kostas Kyriakopoulos
The authors propose a human-robot interface based on potentials recorded through surface Electroencephalographic sensors, aiming to decode human visual attention into motion in three-dimensional space.
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems
Anousheh Gholami, Nariman Torkzaban, John Baras, Chrysa Papagianni
The authors formulate the joint UAV placement and routing problem as a Mixed Integer Linear Program (MILP) and propose an approximation that leads to a LP rounding algorithm and achieves a balance between time-complexity and optimality.
arxiv.org
Zhenyu Lin, John Baras
A modular Q-learning framework to deal with the robot task planning, runtime monitoring and self-correction problem.
2020 American Control Conference
2022
Xiaomin Lin, Nare Karapetyan, Kaustubh Joshi, Tianchen Liu, Nikhil Chopra, Miao Yu, Pratap Tokekar, Yiannis Aloimonos
Autonomous navigation in the underwater environment is challenging due to limited visibility, dynamic changes, and the lack of a cost-efficient accurate localization system. We introduce UIVNAV, a novel end-to-end underwater navigation solution designed to drive robots over Objects of Interest (OOI) while avoiding obstacles, without relying on localization. UIVNAV uses imitation learning and is inspired by the navigation strategies used by human divers who do not rely on localization.
arXiv.org
Gurtajbir Singh Herr, Lasitha Weerakoon, Miao Yu, Nikhil Chopra
The authors utilize an optimization based planner, namely TEB local planner, as the expert to generate high quality motion commands for a car-like robot operating in a simulated dynamic environment. They use these labels to train a deep neural network that learns to navigate. The deep learning-based planner is further augmented with safety margins to enhance its effectiveness in collision avoidance.
Proceedings of the ASME 2022 International Mechanical Engineering Congress and Exposition IMECE2022
Tianchen Liu, Miao Yu, Nikhil Chopra
Navigation of autonomous underwater vehicles (AUVs) is critical for many vital tasks operated in the aquatic environment. However, the navigation problem is challenging in confined environments like caves. Since humans can usually solve such a problem with minimal effort, the main objective of this paper is to implement learning-based neural network models for AUVs to learn from human actions in a confined environment. The Girona underwater dataset, acquired in a cave exploration experiment, was used for the learning task. The odometry and sonar data from the dataset were used to train two deep neural network models for AUV navigation through the underwater caves. The test data demonstrates that both models achieve relatively high accuracy for the navigation task.
IEEE OCEANS 2022 Conference, Hampton Roads, Va.
Lasitha Weerakoon, Gurtajbir Singh Herr, Jasmine Blunt, Miao Yu, Nikhil Chopra
The authors study algorithms for detecting and including glass objects in an optimization-based Simultaneous Localization and Mapping (SLAM) algorithm for safe and reliable robot navigation. They utilize a simple and computationally inexpensive glass detection scheme for detecting glass objects and present the methodology to incorporate the identified objects into the occupancy grid maintained by such an algorithm (Google Cartographer).
arXiv.org
Lasitha Weerakoon, Nikhil Chopra
This paper introduces a novel robotic system, coined as a hybrid rigid-soft (HyRiSo) robot composed of rigid links and soft links serially attached. HyRiSo combines the benefits of the dexterity of a soft manipulator with the support capability of a classical stiff arm.
researchgate.net
Tianchen Liu, Nikhil Chopra, Jayesh Samtani
Many strawberry growers in some areas of the United States rely on customers to pick the fruits during the peak harvest months. Unfavorable weather conditions such as high humidity and excessive rainfall can quickly promote fruit rot and diseases. This study establishes an elementary farm information system to demonstrate timely information on the farm and fruit conditions (ripe, unripe) to the growers. The information system processes a video clip or a sequence of images from a camera to provide a map which can be viewed to estimate quantities of strawberries at different stages of ripeness. The farm map is built by state-of-the-art vision-based simultaneous localization and mapping (SLAM) techniques, which can generate the map and track the motion trajectory using image features. It can help farm labor direct traffic to specific strawberry locations within a farm where fruits need to be picked, or where berries need to be removed. The obtained system can help reduce farm revenue loss and promote sustainable crop production.
Proceedings of the 2022 Biology and Life Sciences Forum
Jeffrey Twigg, Fikadu Dagefu, Nikhil Chopra, Brian Sadler
Safety, security, and rescue missions typically occur in environments where unreliable wireless communication impedes cooperative human and robot missions. A low-VHF band antenna array formed through the coordination of ground robots has the potential to transmit and receive signals more reliably at longer ranges in these environments. The authors advance this idea by developing methods for controlling a robot forming a low-VHF antenna array to improve or maintain wireless communication with other agents in these complex environments. They propose and test methods for orienting this robotic antenna array to increase received signal strength (RSS) in a complex environment and show that creating and actuating this array improves RSS in line-of-sight and non-line-of-sight conditions.
IEEE Robotics and Automation Letters
Prasad Vilas Chanekar, Nikhil Chopra
Empirical experiences have shown that simultaneous (rather than conventional sequential) plant and controller design procedure leads to an improvement in performance and saving of plant resources. Such a simultaneous synthesis procedure is called a “co-design.” This paper studies the co-design problem for a class of Lipschitz nonlinear dynamical systems having a quadratic control objective and state-feedback controller. It proposes time-independent reformulation of a co-design optimization problem whose constraints ensure stability of a system. The authors also present a gradient-based co-design solution procedure which involves system coordinate transformation and whose output is a provably stable solution for the original system. They show the efficacy of the solution procedure through co-design of a single-link robot.
arXiv.org
2021
Lasitha Weerakoon, Nikhil Chopra
The paper introduces a soft robotic system that is a soft inverted pendulum with a revolute joint at the base. The revolute joint is not actuated. The soft body is hypothesized to be of constant curvature and it is actuated. Motivated by swing up controllers for classical underactuated systems, a switching-based swing up and stabilization control of the proposed soft robot system is studied. The authors demonstrate that the swing up control guides the soft robot to the desired energy level, which is the upright position.
researchgate.net
2020
Kushal Chakrabarti, Nirupam Gupta, Nikhil Chopra
This paper considers the multi-agent distributed linear least-squares problem. The authors extend the idea of iterative pre-conditioning to stochastic settings, where the server updates the estimate and the iterative pre-conditioning matrix-based on a single randomly selected data point at every iteration.
arXiv.org
Jeffrey Twigg, Nikhil Chopra, Brian Sadler
Recent developments in low-VHF antenna design and parasitic antenna array research show it is possible to form multi-robot antenna arrays that can extend communication range in urban and indoor environments. This paper proposes a generalized version of a disk model that describes a parasitic array. This model is then integrated into a Fiedler maximization approach for maintaining communication.
IEEE Robotics and Automation Letters
Lasitha Weerakoon, Nikhil Chopra
This paper investigates an adaptive task space bilateral teleoperation framework for soft robots with dynamic uncertainties. It assumes a non-redundant rigid master manipulator and a redundant soft slave manipulator under the piecewise constant curvature hypothesis.
2020 American Control Conference
2019
Jeffrey Twigg, Fikadu Dagefu, Nikhil Chopra, Brian M. Sadler
The paper describes a two-element parasitic array formed by two ground robots and proposes a technique by which this array can optimize its gain in a direction of interest online.
2019 IEEE International Symposium on Safety, Security, and Rescue Robotics
2023
Xiaotian Xu, Guangyao Shi, Pratap Tokekar, Yancy Diaz-Mercado
A distributed interactive framework to provide high-level position instructions for multi-robot aerial cinematography based on coverage over a hemisphere. The control strategy is based on optimization of the coverage functional and geometric relationships over a hemisphere. It enables multiple Unmanned Aerial Vehicles (UAVs) to coordinate their motion while tracking a dynamic (real or virtual) target, and can accommodate high-level human inputs to influence UAV concentration.
2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
2022
Phillip Rivera-Ortiz, Andrew Frommer, Yancy Diaz-Mercado
A verifiable condition for the selection of a sufficient number of pursuers to capture a faster evader. The condition is based on the tracking performance of a multi-agent control scheme.
2022 IEEE 61st Conference on Decision and Control (CDC)
Phillip Rivera, Marin Kobilarov, Yancy Diaz-Mercado
Scalable pursuer coordination for reach-avoid games against a fast evader are developed leveraging coverage control over manifolds. The maintenance of a manifold, termed defense surface, prevents the evader and its target from occupying the same half-space and shown sufficient as a cooperative capture strategy. Nonlinear control synthesis continually reconfigures the pursuers to enable a defense surface via coverage. Simulation results empirically validate that the proposed condition serves as a surrogate objective for pursuer team coordination.
IEEE Transactions on Automatic Control
Phillip Rivera, Marin Kobilarov, Yancy Diaz-Mercado
Scalable pursuer coordination for reach-avoid games against a fast evader are developed leveraging coverage control over manifolds. The maintenance of a manifold, termed defense surface, prevents the evader and its target from occupying the same half-space and shown sufficient as a cooperative capture strategy. Nonlinear control synthesis continually reconfigures the pursuers to enable a defense surface via coverage. Simulation results empirically validate that the proposed condition serves as a surrogate objective for pursuer team coordination.
IEEE Transactions on Automatic Control
Yotam Barnoy, Onder Erin, Suraj Raval, Will Pryor, Lamar O. Mair, Irving N. Weinberg, Yancy Diaz-Mercado, Axel Krieger, Yancy Diaz-Mercado, Gregory Hager
Magnetically manipulated medical robots are a promising alternative to current robotic platforms, allowing for miniaturization and tetherless actuation. Controlling such systems autonomously may enable safe, accurate operation. However, classical control methods require rigorous models of magnetic fields, robot dynamics, and robot environments, which can be difficult to generate. Model-free reinforcement learning (RL) offers an alternative that can bypass these requirements. Here, the authors apply RL to a robotic magnetic needle manipulation system.
IEEE Transactions on Medical Robotics and Bionics
Erick J. Rodríguez-Seda, Xiaotian Xu, Josep M. Olm, Arnau Dòria-Cerezo, Yancy Diaz-Mercado
A distributed asynchronous self-triggered control policy for centroidal Voronoi coverage control that is shown to decrease the sampling or communication instants among agents without degrading the performance of the mobile sensor network.
IEEE Transactions on Robotics
Onder Erin, Suraj Raval, Trevor J. Schwehr, Will Pryor, Yotam Barnoy, Adrian Bell, Xiaolong Liu, Lamar Mair, Irving Weinberg, Axel Krieger, Yancy Diaz-Mercado
Magnetic actuation holds promise for wirelessly controlling small, magnetic surgical tools and may enable the next generation of ultra minimally invasive surgical robotic systems. Precise torque and force exertion are required for safe surgical operations and accurate state control. Dipole field estimation models perform well far from electromagnets but yield large errors near coils. Thus, manipulations near coils suffer from severe (10x) field modeling errors. The authors experimentally quantify closed-loop magnetic agent control performance by using both a highly erroneous dipole model and a more accurate numerical magnetic model to estimate magnetic forces and torques for any given robot pose in 2D. Such improved magnetic modelling is crucial for systems requiring robust estimates of magnetic forces for positioning agents, particularly in force-sensitive environments like surgical manipulation.
2022 IEEE Robotics and Automation Letters
Sarjana Oradiambalam Sachidanandam, Sara Honarvar, Yancy Diaz-Mercado
Human-Swarm Interaction (HSI) is a fast-growing research area in swarm robotics. The researchers investigate the effectiveness of augmented reality-based interfaces as tools for remote interaction in HSI systems. They develop a simple AR-based interface and evaluate its effectiveness against an unaugmented interface, using remote human user studies.
2022 International Conference on Robotics and Automation (ICRA)
2021
M. Marge, C. Espy-Wilson, N. Ward, A. Alwan, Y. Artzi, M. Bansal, G. Blankenship, J. Chai, H. Daumé III, D. Dey , M. Harper, T.Howard, C. Kennington I. Kruijff-Korbayová, D. Manocha, C. Matuszek, R. Mead, R.Mooney, R. K. Moore, M. Ostendorf, H. Pon-Barry, A. Rudnicky, M. Scheutz, R. St. Amant, T. Sun, S. Tellex, D. Traum, Z. Yu
With robotics rapidly advancing, more effective human–robot interaction is increasingly needed to realize the full potential of robots for society. While spoken language must be part of the solution, our ability to provide spoken language interaction capabilities is still very limited. In this article, based on the report of an interdisciplinary workshop convened by the National Science Foundation, the authors identify key scientific and engineering advances needed to enable effective spoken language interaction with robotics. The authors make 25 recommendations, involving eight general themes: putting human needs first, better modeling the social and interactive aspects of language, improving robustness, creating new methods for rapid adaptation, better integrating speech and language with other communication modalities, giving speech and language components access to rich representations of the robot’s current knowledge and state, making all components operate in real time, and improving research infrastructure and resources. Research and development that prioritizes these topics will, we believe, provide a solid foundation for the creation of speech-capable robots that are easy and effective for humans to work with.
Computer Speech & Language, January 2022
2023
Nitin Sanket, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos
Robots are active agents that operate in dynamic scenarios with noisy sensors. Predictions based on these noisy sensor measurements often lead to errors and can be unreliable. To this end, roboticists have used fusion methods using multiple observations. Lately, neural networks have dominated the accuracy charts for percep- tion-driven predictions for robotic decision-making and often lack uncertainty metrics associated with the pre- dictions. Here, the authors present a mathematical formulation to obtain the heteroscedastic aleatoric uncertainty of any arbitrary distribution without prior knowledge about the data. The approach has no prior assumptions about the prediction labels and is agnostic to network architecture.
Science Robotics
Snehesh Shrestha, Ge Gao, Cornelia Fermuller, Yiannis Aloimonos
About this paper, the authors write: "Many of us researchers take extra measures to control for known-unknowns. However, unknown-unknowns can, at best, be negligible, but otherwise, they could produce unreliable data that might have dire consequences in real-life downstream applications. Human-Robot Interaction standards informed by empirical data could save us time and effort and provide us with the path toward the robots of the future. To this end, we share some of our pilot studies, lessons learned, and how they affected the outcome of our experiments. While these aspects might not be publishable in themselves, we hope our work might save time and effort for other researchers towards their research and serve as additional considerations for discussion at the workshop."
github.io
Snehesh Shrestha, Yantian Zha, Ge Gao, Cornelia Fermuller, Yiannis Aloimonos
The work introduces NatSGD, a multimodal HRI dataset that contains human commands as speech and gestures, along with robot behavior in the form of synchronized demonstrated robot trajectories. These data enable HRI with Imitation Learning so that robots can learn to work with humans in challenging, real-life domains such as performing complex tasks in the kitchen.
github.io
2021
EVPropNET: Detecting drone by finding propellers for mid-air landing and following
Yiannis Aloimonos, Cornelia Fermüller, Nitin Sanket, Chahat Deep Singh, Chethan Parameshwara, Guido C.H.E. de Croon
The authors use the most ubiquitous part of a drone—the propeller—as the basis for a new detection scheme that is 92% accurate.
2021 Robotics Science and Systems Conference (RSS 2021)
2020
Nitin J. Sanket, Chahat Deep Singh, Varun Asthana, Cornelia Fermüller, Yiannis Aloimonos
A framework for quadrotor navigation based on a stereo camera system whose baseline can be adapted on the fly.
arXiv.org
Chengxi Ye, Anton Mitrokhin, Cornelia Fermüller, James Yorke, Yiannis Aloimonos
Presents an unsupervised learning pipeline for dense depth, optical flow and egomotion estimation for autonomous driving applications, using the event-based output of the Dynamic Vision Sensor as input.
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Kanishka Ganguly, Behzad Sadrfaridpour, Krishna Bhavithavya Kidambi, Cornelia Fermüller, Yiannis Aloimonos
An adaptive,compliant grasping strategy for robots using only tactile feedback.
arXiv.org
Nitin J. Sanket, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos
The authors present a simple way to estimate ego-motion/odometry on an aerial robot using deep learning combining commonly found on-board sensors: a up/down-facing camera, an altimeter source and an IMU.
arXiv.org
Nitin J. Sanket, Chethan M. Parameshwara, Chahat Deep Singh, Ashwin V. Kuruttukulam, Cornelia Fermüller, Davide Scaramuzza, Yiannis Aloimonos
A deep learning-based solution for dodging multiple dynamic obstacles on a quadrotor with a single event camera and on-board computation.
2020 IEEE International Conference on Robotics and Automation (ICRA)
2021
Seyede Fatemeh Ghoreishi, Ryan Sochol, Dheeraj Gandi, Axel Krieger, Mark Fuge
The authors propose modeling multi-actuator soft catheters to enable alignment with desired vessel shapes near the target area. They develop mathematical models to simulate the catheter’s positioning due to the moments exerted by multiple pneumatic actuators along the catheter and use those models to compare optimization approaches that can achieve catheter alignment along a desired vessel shape.
IEEE Transactions on Medical Robotics and Bionics
2023
Alexandra Shaver, Calvin Lu, Christopher Gaskins, James Reggia, James Purtilo, Rodolphe J. Gentili
The development and use of virtual systems to learn and perform remotely complex action sequences has many applications from rehabilitation to military training. The examination of human cognitive-motor control and learning processes in individuals executing a task physically in-person and remotely via a virtual system can help with the design of such systems as well as inform human performance. An important application of such virtual systems is the remote assessment of cognitive-motor performance and learning for the neurorehabilitation of individuals with neural deficits (e.g., veterans). This study is a first step that examined performance, mental workload, and fatigue in healthy individuals during remote practice of action sequences compared to physical in-person training.
2023 11th International IEEE/EMBS Conference on Neural Engineering (NER)
2022
Gregory Davis, Garrett Katz, James Reggia, Rodolphe Gentili
The researchers explore the viability of developing purely-neural controllers for social robots that learn to imitate by reasoning about the underlying intentions of demonstrated behaviors. NeuroCERIL is a brain-inspired neurocognitive architecture that uses a novel hypothetico-deductive reasoning procedure to produce generalizable and human-readable explanations for demonstrated behavior.
arXiv.org
2023
Sidney Molnar, Matt Mueller, Robert Macpherson, Lawrence Rhoads, Jeffrey Herrmann
This paper describes metareasoning approaches that enable a robot to select a new path planning algorithm when the current planning algorithm cannot find a feasible solution.
UMD DRUM Library
Akshay Bapat, Bharath Reddy Bora, Jeffrey Herrmann, Shapour Azarm, Huan Xu, Michael Otte
Explores the problem of task allocation among networked multi-robot systems when communication conditions are poor. The authors develop solutions that ensure all tasks are eventually completed—even if some agents in the network are destroyed. Their two new task allocation algorithms assume communication may not happen, but benefit the robotic agents and their missions whenever communication is successful.
IEEE Access
2020
Ruchir Patel, Eliot Rudnick-Cohen, Shapour Azarm, Michael Otte, Huan Xu, Jeffrey W. Herrmann
In multi-agent collaborative search missions, task allocation is required to determine which agents will perform which tasks. The paper proposes a new approach for decentralized task allocation based on a decentralized genetic algorithm.
2020 International Conference on Robotics and Automation
Sharan Nayak, Mohamed Khalid M. Jaffar, Estefany Carrillo, Suyash Yeotikar, Eliot Rudnick-Cohen, Ruchir Patel, Jeffrey Herrmann, Huan Xu, Shapour Azarm, Michael Otte
An experimental comparison of the performance of five state-of-the-art decentralized task allocation algorithms under imperfect communication conditions for teams of unmanned aerial vehicles (UAVs).
IEEE Robotics and Automation Letters
Jeffrey Herrmann
A novel data-driven metareasoning approach that generates a metareasoning policy that agents in a multi-agent system can use whenever they must collaborate to assign tasks.
ISR Technical Report, DRUM
2020
Alexander Castro, Leo Peckerar, Timothy Horiuchi, Pamela Abshire
A nano-quadcopter system that performs odor detection in a windless indoor environment. The system uses the Crazyflie platform and incorporates a single commercially available metal oxide semiconductor gas sensor. The authors demonstrate that rotor-induced airflow enhances odor detection. It can be considered to be the quadrotor equivalent of sniffing behavior.
2020 IEEE SENSORS Conference
2023
Udit Halder, Vidya Raju, Matteo Mischiati, Biswadip Dey, P. S. Krishnaprasad
Avian flocks display a wide variety of flight behaviors, including steady directed translation of center of mass, rapid change of overall morphology, re-shuffling of positions of individuals within a persistent form, etc. These behaviors may be viewed as flock-scale strategies, emerging from interactions between individuals, accomplishing some collective adaptive purpose such as finding a roost, or mitigating the danger from predator attacks. While we do not conceive the flock as a single cognitive agent, the moment-to-moment decisions of individuals, influenced by their neighbors, appear as if to realize collective strategies that are cognizant of purpose. In this paper, we identify the actions of the flock as allocation of energetic resources, and thereby associate a cognitive cost to behavior. Our notion of cognitive cost is a measure of the temporal variability of such resource allocation. Using a recently developed natural geometric approach to kinetic energy allocation, we map the flock behavior to a temporal signature on the standard (probability) simplex. From the signature of a flocking event, we calculate the cognitive cost as a solution to an optimal control problem based on a game-theoretic model. Alternatively, one can associate to a signature an entropic cost. These two-cost measures, when applied to data on starling flocks, show a consistent spread in value across events, and we suggest the possibility that higher cost may arise from flock response to predator attacks.
Systems & Control Letters
Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Mohamed Elnoor, Dinesh Manocha
AdVENTR is a new system for autonomous robot navigation in unstructured outdoor environments with uneven and vegetated terrains. The general approach can enable both wheeled and legged robots to handle outdoor terrain complexity including unevenness, surface properties like poor traction, granularity, obstacle stiffness, etc. The system uses data from sensors including RGB cameras, 3D Lidar, IMU, robot odometry, and pose information with efficient learning-based perception and planning algorithms that can execute on edge computing hardware. AdVENTR uses a scene-aware switching method to perceive the environment for navigation at any time instant and dynamically switches between multiple perception algorithms.
arXiv.org
Chak Lam Shek, Xiyang Wu, Dinesh Manocha, Pratap Tokekar, and Amrit Singh Bedi
Optimal locomotion policy can be context-dependent by using the contextual information of encountered terrains in decision-making. Humans can interpret the environmental context for robots, but the ambiguity of human language makes it challenging to use in robot locomotion directly. In this paper, the authors propose a novel approach, LANCAR, that introduces a context translator that works with reinforcement learning (RL) agents for context-aware locomotion. The formulation allows a robot to interpret the contextual information from environments generated by human observers or Vision-Language Models (VLM) with Large Language Models (LLM) and use this information to generate contextual embeddings.
arXiv.org
Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Mohamed Elnoor, Dinesh Manocha
VAPOR is a new method for autonomous legged robot navigation in unstructured, densely vegetated outdoor environments using offline Reinforcement Learning (RL).
arXiv.org
Jing Liang, Peng Gao, Xuesu Xiao, Adarsh Jagan Sathyamoorthy,Mohamed Elnoor, Ming Lin, Dinesh Manocha
A learning algorithm for trajectory generation for outdoor robot navigation. The goal is to compute collision-free paths that also satisfy environment-specific traversability constraints.
arXiv.org
Adarsh Jagan Sathyamoorthy, Kasun Weerakoon, Mohamed Elnoor, Dinesh Manocha
The authors present Multi-Layer Intensity Map, a novel 3D object representation for robot perception and autonomous navigation. It consists of multiple stacked layers of 2D grid maps each derived from reflected point cloud intensities corresponding to a certain height interval. The different layers of the intensity maps can be used to simultaneously estimate obstacles’ height, solidity/density, and opacity. The work demonstrates that they can help accurately differentiate obstacles that are safe to navigate through from ones that must be avoided in indoor and outdoor environments.
arXiv.org
Senthil Hariharan Arul, Dinesh Manocha
CGLR is a decentralized path-planning algorithm for navigating multiple differential-drive robots in dense environments. In contrast to prior decentralized methods, here the authors propose a novel congestion metric-based replanning that couples local and global planning techniques to efficiently navigate in scenarios with multiple corridors.
2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Mohamed Elnoor, Adarsh Jagan Sathyamoorthy, Kasun Weerakoon, Dinesh Manocha
ProNav uses proprioceptive signals for traversability estimation in challenging outdoor terrains for autonomous legged robot navigation.
arXiv.org
Jiangbei Yue, Dinesh Manocha, He Wang
Human trajectory forecasting helps to understand and predict human behaviors, enabling applications from social robots to self-driving cars, and therefore has been heavily investigated. Most existing methods can be divided into model-free and model-based methods. Model-free methods offer superior prediction accuracy but lack explainability, while model-based methods provide explainability but cannot predict well. Combining both methodologies, the authors propose a new Bayesian Neural Stochastic Differential Equation model BNSP-SFM, where a behavior SDE model is combined with Bayesian neural networks (BNNs).
arXiv.org
Souradip Chakraborty, Amrit Singh Bedi, Kasun Weerakoon, Prithvi Poddar, Alec Koppel, Pratap Tokekar, Dinesh Manocha
A novel Heavy-Tailed Stochastic Policy Gradient (HT-PSG) algorithm to deal with the challenges of sparse rewards in continuous control problems.
2023 IEEE International Conference on Robotics and Automation (ICRA)
Divya Kothandaraman, Ming Lin, Dinesh Manocha
DifFAR is a learning algorithm for human activity recognition in videos. The approach is designed for UAV videos, which are mainly acquired from obliquely placed dynamic cameras that contain a human actor along with background motion. Typically, the human actors occupy less than one-tenth of the spatial resolution. DifFAR simultaneously harnesses the benefits of frequency domain representations, a classical analysis tool in signal processing, and data driven neural networks. The authors build a differentiable static-dynamic frequency mask prior to model the salient static and dynamic pixels in the video, crucial for the underlying task of action recognition.
2023 IEEE International Conference on Robotics and Automation (ICRA)
Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Dinesh Manocha
A method to perceive a terrain’s geometric and other surface properties for efficient motion planning in outdoor environments. The method incorporates two perception branches to identify the terrain’s elevation and roughness separately. The first branch uses an elevation map created using LiDAR point clouds to compute a cost map that represents critical elevation changes. The second branch uses a vision-based cost map trained using RGB images, IMU, and robot odometry. Then, least-cost waypoints are calculated on a combined cost map and are followed using the Dynamic Window Approach (DWA). This planner navigates along the least-cost waypoints while adaptively varying the velocities to reduce vibration. The authors evaluate the method’s performance on a Husky robot in real-world environments and observe that the method leads to higher success rates, and lower vibrations compared to state-of-the-art methods.
SPIE Proceedings of the Open Architecture/Open Business Model Net-Centric Systems and Defense Transformation 2023
Divya Kothandaraman, Xijun Wang, Tianrui Guan, Sean Hu, Ming Lin, Dinesh Manocha
Aerial video recognition is challenging due to various factors. Prior work on action recognition imposes constraints in terms of unavailability of object detection bounding box ground-truth inhibiting the application of localization models and computational constraints preventing the usage of expensive space-time self-attention. Optical flow and pretrained models for detecting human actor performing action do not work too well due to domain gap issues. Here, the authors present a frequency-domain space-time attention method that encapsulates long-range space-time dependencies by emulating the weighted outer product in the frequency domain. They also show a frequency-based object background disentanglement method to inherently separate out the moving human actor from the background and a mathematical model for static salient regions and an identity loss function to learn disentangled features in a differentiable manner.
SPIE Proceedings of the Open Architecture/Open Business Model Net-Centric Systems and Defense Transformation 2023
Ruiqi Xian, Xijun Wang, Divya Kothandarama, Dinesh Manocha
A new algorithm for selection of informative frames in video action recognition. The approach is designed for aerial videos captured using a moving camera where human actors occupy a small spatial resolution of video frames. The algorithm utilizes the motion bias within aerial videos, which enables the selection of motion-salient frames. The authors introduce the concept of patch mutual information (PMI) score to quantify the motion bias between adjacent frames, by measuring the similarity of patches.
arXiv.org
Vrushabh Zinage, Senthil Hariharan Arul, Dinesh Manocha, Satadal Ghosh
An online motion planning algorithm (3D-OGSSE) for generating smooth, collision-free trajectories over multiple planning iterations for a 3-D agent operating in an unknown, obstacle-cluttered, 3-D environment. In each planning iteration, 3D-OGSSE constructs an obstacle-free region termed “generalized sensed shape” based on the locally-sensed environment information and the notion of generalized shape. A collision-free path is computed by sampling points in the generalized sensed shape and is used to generate a smooth, time-parametrized trajectory by minimizing snap. The generated trajectory at every planning iteration is constrained to lie within generalized sensed shape, which ensures the agent maneuvers in locally obstacle-free space. As the agent reaches the boundary of the generalized sensed shape in a planning iteration, a re-plan is triggered by a receding horizon planning mechanism that also enables the initialization of the next planning iteration.
IEEE Robotics and Automation Letters
Tianrui Guan, Aswath Muthuselvam, Montana Hoover, Xijun Wang, Jing Liang, Adarsh Jagan Sathyamoorthy, Damon Conover, Dinesh Manocha
This is a novel 3D place recognition method that solves a large-scale point matching problem in a cross-source setting. Cross-source point cloud data corresponds to point sets captured by depth sensors with different accuracies or from different distances and perspectives. We address the challenges in terms of developing 3D place recognition methods that account for the representation gap between points captured by different sources. Our method handles cross-source data by utilizing multi-grained features and selecting convolution kernel sizes that correspond to most prominent features.
arXiv.org
Adarsh Jagan Sathyamoorthy, Kasun Weerakoon, Tianrui Guan, Mason Russell, Damon Conover, Jason Pusey, Dinesh Manocha
This is a method for autonomous legged robot navigation in densely vegetated environments with a variety of pliable/traversable and non-pliable/untraversable vegetation. The authors present a few-shot learning classifier that can be trained on a few hundred RGB images to differentiate flora that can be navigated through, from the ones that must be circumvented. Using the vegetation classification and 2D lidar scans, this method constructs a vegetation-aware traversability cost map that accurately represents the pliable and non-pliable obstacles with lower, and higher traversability costs, respectively. The cost map construction accounts for misclassifications of the vegetation and further lowers the risk of collisions, freezing and entrapment in vegetation during navigation.
arXiv.org
Liang He, Zherong Pan, Dinesh Manocha
A lightweight, decentralized algorithm for navigating multiple nonholonomic agents through challenging environments with narrow passages. The key idea is to allow agents to yield to each other in large open areas instead of narrow passages, to increase the success rate of conventional decentralized algorithms.
arXiv.org
Souradip Chakraborty, Kasun Weerakoon Prithvi Poddar, Pratap Tokekar, Amrit Singh Bedi, Dinesh Manocha
Reinforcement learning-based policies for continuous control robotic navigation tasks often fail to adapt to changes in the environment during real-time deployment, which may result in catastrophic failures. To address this limitation, the authors propose a novel approach called RE-MOVE (REquest help and MOVE on), which uses language-based feedback to adjust trained policies to real-time changes in the environment.
arXiv.org
Ruichen Wang, Dinesh Manocha
A novel ray tracing-based radio propagation algorithm that can handle large urban scenes with hundreds or thousands of dynamic objects and receivers.
arXiv.org
Senthil Hariharan Arul, Jong Jin Park, Dinesh Manocha
An algorithm for safe robot navigation in complex dynamic environments using a variant of model predictive equilibrium point control.
arXiv.org
Steve Tonneau, Andrea Del Prete, Julien Pettre, Chonhyon Park, Dinesh Manocha, Nicolas Mansard
A framework capable of producing contact plans describing complex multiped motions (including humanoid): standing up, climbing stairs using a handrail, crossing rubble and getting out of a car. The framework answers a need demonstrated at the DARPA Robotics Challenge, where the lack of an automatic acyclic contact planner was recognized as a major issue. Here the novel key idea is the reachability condition.
HAL archive
Vishnu Sashank Dorbala, James Mullen, Dinesh Manocha
LGX is a novel algorithm for Object Goal Navigation in a "language-driven, zero-shot manner", where an embodied agent navigates to an arbitrarily described target object in a previously unexplored environment. The approach leverages the capabilities of Large Language Models (LLMs) for making navigational decisions by mapping the LLM's implicit knowledge about the semantic context of the environment into sequential inputs for robot motion planning.
arXiv.org
Xijun Wang, Ruiqi Xian, Tianrui Guan, Celso de Melo, Stephen Nogar, Aniket Bera, Dinesh Manocha
A new approach for aerial video action recognition that is designed for videos captured using UAVs and can run on edge or mobile devices. The learning-based approach uses customized auto zoom to automatically identify the human target and scale it appropriately. This makes it easier to extract key features and reduces the computational overhead.
arXiv.org
Ruiqi Xian, Xijun Wang, Dinesh Manocha
This paper presents a novel approach for action recognition in UAV videos. The formulation is designed to handle occlusion and viewpoint changes caused by the movement of a UAV. The researchers use the concept of mutual information to compute and align the regions corresponding to human action or motion in the temporal domain. This enables the recognition model to learn from the key features associated with the motion.
arXiv.org
2022
Yu-Ping Wang, Hao-Ning Wang, Zi-Xin Zou, Dinesh Manocha
Communication between robots and the server is a major problem for cloud robotic systems. The authors address the problem caused by data loss during such communications, and propose an efficient buffering algorithm, AFR, to solve the problem.
2022 International Conference on Intelligent Robots and Systems
Yuzhen Zhang, Wentong Wang, Weizhi Guo, Pei Lv, Mingliang Xu, Wei Chen, Dinesh Manocha
A profound understanding of inter-agent relationships and motion behaviors is important to achieve high-quality planning when navigating in complex scenarios, especially at urban traffic intersections. This paper presents D2-TPred, a trajectory prediction approach with respect to traffic lights, which uses a spatial dynamic interaction graph (SDG) and a behavior dependency graph (BDG) to handle the problem of discontinuous dependency in the spatial-temporal space. Specifically, the SDG is used to capture spatial interactions by reconstructing sub-graphs for different agents with dynamic and changeable characteristics during each frame.
ECCV 2022
Liang He, Zherong Pan, Kiril Solovey, Biao Jia, Dinesh Manocha
Here is a centralized algorithm for labeled, disk-shaped Multi-Robot Path Planning (MPP) in a continuous planar workspace with polygonal boundaries. The method automatically transform the continuous problem into a discrete, graph-based variant termed the pebble motion problem, which can be efficiently solved.
GAMMA Lab papers archive
Aaron M. Roth, Jing Liang, Ram Sriram, Elham Tabassi, Dinesh Manocha
The paper introduces Multiple Scenario Verifiable Reinforcement Learning via Policy Extraction (MSVIPER), a new method for policy distillation to decision trees for improved robot navigation. MSVIPER learns an “expert” policy using any Reinforcement Learning (RL) technique involving learning a state-action mapping and then uses imitation learning to learn a decision-tree policy from it. MSVIPER results in efficient decision trees and can accurately mimic the behavior of the expert policy.
arXiv.org
Divya Kothandaraman, Ming Lin, Dinesh Manocha
A learning algorithm for human activity recognition in videos. The approach is designed for UAV videos, which are mainly acquired from obliquely placed dynamic cameras that contain a human actor along with background motion.
arXiv.org
Sanghyun Son, Jing Liang, Ming Lin, Dinesh Manocha
The paper presents a novel differentiable weighted generalized iterative closest point (WGICP) method applicable to general 3D point cloud data, including that from lidar. This method builds on differentiable generalized ICP (GICP) and uses the differentiable K-Nearest Neighbors (KNN) algorithm to enhance differentiability. Our differentiable GICP algorithm provides the gradient of output pose estimation with respect to each input point. This allows the researchers to train a neural network to predict its importance, or weight, in order to estimate the correct pose.
arXiv.org
Divya Kothandaraman, Tianrui Guan, Xijun Wang, Shuowen Hu, Ming Lin, Dinesh Manocha
The algorithm Fourier Activity Recognition (FAR) is presented for UAV video activity recognition. The formulation uses a novel Fourier object disentanglement method to innately separate out the human agent (which is typically small) from the background. The disentanglement technique operates in the frequency domain to characterize the extent of temporal change of spatial pixels, and exploits convolution-multiplication properties of Fourier transform to map this representation to the corresponding object-background entangled features obtained from the network. This formulation uses much fewer computations than self-attention and has been evaluated on multiple UAV datasets including UAV Human RGB, UAV Human Night, Drone Action, and NEC Drone.
Computer Vision ECCV 2022 (17th European Conference)
Kasun Weerakoon, Souradip Chakraborty, Nare Karapetyan, Adarsh Jagan Sathyamoorthy, Amrit Singh Bedi, Dinesh Manocha
An approach to improve the performance of deep reinforcement learning (DRL) based outdoor robot navigation systems that uses easy-to-design parse rewards. The authors propose a novel adaptive Heavy-Tailed Reinforce algorithm for Outdoor Navigation called HTRON. The main idea is to utilize heavy-tailed policy parametrizations which implicitly induce exploration in sparse reward settings.
arXiv.org
Senthil Hariharan Arul, Dinesh Manocha
Multi-agent navigation methods for autonomous robots maneuvering in complex environments. The approach considers the agent’s dynamics to compute smooth paths that avoid collisions with other agents and static and dynamic obstacles in a decentralized fashion.
Proceedings of SPIE 12113, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications IV
Senthil Hariharan Arul, Amrit Singh Bedi, Dinesh Manocha
Agents in decentralized multi-agent navigation lack the world knowledge to make safe and (near-)optimal plans reliably. They base their decisions on their neighbors' observable states, which hide the neighbors' navigation intent. Here, the authors propose augmenting decentralized navigation with inter-agent communication to improve their performance and aid an agent in making sound navigation decisions. They present a reinforcement learning method for multi-agent collision avoidance using selective inter-agent communication.
Cornell University arXiv.org
Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Jing Liang, Tianrui Guan, Utsav Patel, Dinesh Manocha
A trajectory traversability estimation and planning algorithm for robot navigation in complex outdoor environments. The algorithm incorporates multimodal sensory inputs from an RGB camera, 3D LiDAR, and robot’s odometry sensor to train a prediction model to estimate candidate trajectories’ success probabilities based on partially reliable multi-modal sensor observations.
arXiv.org
Aakriti Agrawal, Amrit Singh Bedi, Dinesh Manocha
Here is a reinforcement learning-based algorithm for multi-robot task allocation problem in warehouse environments. It is formulated as a Markov Decision Process and solves via a novel deep multi-agent reinforcement learning method called RTAW with attention inspired policy architecture.
arXiv.org
Aakriti Agrawal, Senthil Hariharan, Amrit Singh Bedi, Dinesh Manocha
This is a reinforcement learning (RL) based task allocation and decentralized navigation algorithm for mobile robots in warehouse environments. The approach is designed for scenarios in which multiple robots are used to perform various pick up and delivery tasks.
arXiv.org
Qiaoyun Wu, Jun Wang, Jing Liang, Xiaoxi Gong, Dinesh Manocha
The authors use a four-module approach for image-goal navigation, where an agent navigates with a goal image rather than accurate target information, which is more challenging. The work decouples the learning of navigation goal planning, collision avoidance, and navigation ending prediction, enabling more concentrated learning of each part.
IEEE Robotics and Automation Letters
Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Dinesh Manocha
Deep Reinforcement Learning (DRL) is hugely successful due to the availability of realistic simulated environments. However, performance degradation during simulation to real-world transfer still remains a challenging problem for the policies trained in simulated environments. To close this sim-to-real gap, we present a novel hybrid architecture that utilizes an intermediate output from a fully trained attention DRL policy as a navigation cost map for outdoor navigation.
arXiv.org
Jing Liang, Kasun Weerakoon, Tianrui Guan, Nare Karapetyan, Dinesh Manocha
A novel outdoor navigation algorithm to generate stable and efficient actions to navigate a robot to the goal. The authors use a multi-stage training pipeline to show their model produces policies that result in stable and reliable robot navigation on complex terrains. Based on the Proximal Policy Optimization (PPO) algorithm, they developed a novel method to achieve multiple capabilities for outdoor navigation tasks, namely: alleviating the robot’s drifting, keeping the robot stable on bumpy terrains, avoiding climbing on hills with steep elevation changes, and collision avoidance.
IEEE Robotics and Automation Letters
Rohan Chandra, Mingyu Wang, Mac Schwager, Dinesh Manocha
A novel approach for risk-aware planning with human agents in multi-agent traffic scenarios. The approach takes into account the wide range of human driver behaviors on the road, from aggressive maneuvers like speeding and overtaking, to conservative traits like driving slowly and conforming to the right-most lane.
arXiv.org
Adarsh Jagan Sathyamoorthy, Kasun Weerakoon, Tianrui Guan, Jing Liang, Dinesh Manocha
TerraPN is a novel method that learns the surface properties (traction, bumpiness, deformability, etc.) of complex outdoor terrains directly from robot-terrain interactions through self-supervised learning, and uses it for autonomous robot navigation.
2022 International Conference on Intelligent Robots and Systems
Divya Kothandaraman, Tianrui Guan, Xijun Wang, Sean Hu, Ming Lin, Dinesh Manocha
Describes the new Fourier Activity Recognition computer vision algorithm for UAV video activity recognition. The formulation uses a novel Fourier object disentanglement method to innately separate out the human agent (which is typically small) from the background.
arXiv.org
Jaehoon Choi, Dongki Jung, Yonghan Lee, Deokhwa Kim, Dinesh Manocha, Donghwan Lee
A learning algorithm that leverages monocular simultaneous localization and mapping (SLAM) with proprioceptive sensors. Such monocular SLAM systems can provide metrically scaled camera poses.
arXiv.org
Nilesh Suriyarachchi, Faizan Tariq, Christos Mavridis, John Baras, Dinesh Manocha
Intersections without traffic signals are the sites of 40% of all crashes, 50% of serious collisions, and 20% of fatalities. When motorists approach these intersections, which often have “four-way” stop signs, and feature “unprotected” left turns, they must make choices. If two vehicles approach at approximately the same time, or a driver decides not to stop or wait their turn, safety is at issue. In the future, autonomous vehicles equipped with vehicle-to-infrastructure (V2I) communication may be able to provide solutions for handling these scenarios. Such vehicles, called connected autonomous vehicles (CAVs), will be able to provide multiple sensing and actuation points in the traffic flow. GAMEOPT is a novel hybrid approach to cooperative intersection control for dynamic, multi-lane, unsignalized intersections.
arXiv.org
2021
Dawei Wang, Lingping Gao, Ziquan Lan, Wei Li, Jiaping Ren, Jiahui Zhang, Peng Zhang, Pei Zhou, Shengao Wang, Jia Pan, Dinesh Manocha, Ruigang Yang
The paper introduces an intelligent self-driving truck system that includes a traffic simulation module for generating realistic traffic flow; a high-fidelity truck model designed and evaluated for mimicking real truck response; and an intelligent planning module with a learning-based decision making algorithm and multi-mode trajectory planner that takes into account the truck’s constraints, road slope changes, and surrounding traffic flow.
arXiv.org
Adarsh Jagan Sathyamoorthy, Utsav Patel, Moumita Paul, Yash Savle, Dinesh Manocha
A method to automatically detect pairs of humans in a crowded scenario and determine whether they are within about two meters of space of each other using an autonomous mobile robot equipped with closed-circuit television (CCTV) cameras.
PLOS One
Hui Miao, Feixiang Lu, Zongdai Liu, Liangjun Zhang, Dinesh Manocha and Bin Zhou
A new approach to robustly detect and perceive vehicles in different camera views as part of a cooperative vehicle-infrastructure system (CVIS).
ICCV 2021, Computer Vision Foundation
Divya Kothandaraman, Rohan Chandra, Dinesh Manocha
An unsupervised adaptation approach for visual scene understanding in unstructured traffic environments.
ICCV 2021 workshop paper, Computer Vision Foundation
Kasun Weerakoon, Adarsh Jagan Sathyamoorthy, Utsav Patel, Dinesh Manocha
This new method for reliable robot navigation in uneven outdoor terrains employs a novel fully-trained Deep Reinforcement Learning (DRL) network that uses elevation maps of the environment, robot pose, and goal as inputs to compute an attention mask of the environment.
arXiv.org
Tianrui Guan, Zhenpeng He, Dinesh Manocha, Liangjun Zhang
Terrain Traversability Mapping (TTM) is a real-time mapping approach for terrain traversability estimation and path planning for autonomous excavators in an unstructured environment.
arXiv.org
Rohan Chandra, Mridul Mahajan, Rahul Kala, Rishitha Palugulla, Chandrababu Naidu, Alok Jain, Dinesh Manocha
METEOR is a new, complex traffic dataset that captures traffic patterns in unstructured scenarios in India. METEOR consists of more than 1000 one-minute video clips, over 2 million annotated frames with ego-vehicle trajectories, and more than 13 million bounding boxes for surrounding vehicles or traffic agents. METEOR is a unique dataset in terms of capturing the heterogeneity of microscopic and macroscopic traffic characteristics.
arXiv.org
Cong Wang, Yu-Ping Wang, Dinesh Manocha
MotionHint is a self-supervised algorithm for monocular visual odometry (VO) that takes motion constraints into account.
ADS Labs, Harvard
Zherong Pan1, Min Liu, Xifeng Gao, Dinesh Manocha
A pivotal challenge faced by a robot designer is to determine the “optimal” design to accomplish a given task. This paper addresses the problem of computational task-driven, design optimization for planar linkages.
arXiv.org
Rohan Chandra, Dinesh Manocha
GamePlan is a new method for multi-agent planning involving human drivers and autonomous vehicles (AVs) for unsignaled intersections and roundabouts. It also is designed for merging situations.
arXiv.org
Adarsh Jagan Sathyamoorthy, Utsav Patel, Moumita Paul, Nithish K Sanjeev Kumar, Yash Savle, Dinesh Manocha
CoMet is a new social navigation algorithm to compute a group's cohesion and use that to improve a robot's navigation in crowded scenes.
SAO/NASA Astrophysics Data System
Qiaoyun Wu, Jun Wang, Jing Liang, Xiaoxi Gong, Dinesh Manocha
This work studies the problem of image-goal navigation, which entails guiding robots with noisy sensors and controls through real crowded environments.
arXiv.org
Jing Liang, Yi-Ling Qiao, Tianrui Guan, Dinesh Manocha
A modified velocity-obstacle (VO) algorithm that uses probabilistic partial observations of the environment to compute velocities and navigate a robot to a target. The system uses commodity visual sensors, including a mono-camera and a 2D Lidar, to explicitly predict the velocities and positions of surrounding obstacles through optical flow estimation, object detection, and sensor fusion.
IEEE Robotics and Automation Letters
Tianrui Guan, Jun Wang, Shiyi Lan, Rohan Chandra, Zuxuan Wu, Larry Davis, Dinesh Manocha
M3DETR is a new architecture for 3D object detection. It combines different point cloud representations (raw, voxels, bird-eye view) with different feature scales based on multi-scale feature pyramids. Supplemental materials here.
arXiv.org; also presented at WACV 2022 (IEEE Computer Vision Society Winter Conference on Applications of Computer Vision)
Aaron Roth, Jin Liang, Dinesh Manocha
A novel sensor-based learning navigation algorithm to compute a collision-free trajectory for a robot in dense and dynamic environments with moving obstacles or targets.
arXiv.org
Qiaoyun Wu, Kai Xu, Jun Wang, Mingliang Xu, Xiaoxi Gong, Dinesh Manocha
To enhance the cross-target and cross-scene generalization of target-driven visual navigation based on deep reinforcement learning (RL), the authors introduce an information-theoretic regularization term into the RL objective. The regularization maximizes the mutual information between navigation actions and visual observation transforms of an agent, thus promoting more informed navigation decisions.
IEEE Robotics and Automation Letters
Angelos Mavrogiannis, Rohan Chandra, Dinesh Manocha
An algorithm for behaviorally-guided action prediction and local navigation for autonomous driving in dense traffic scenarios.
IEEE Robotics and Automation Letters 2022
Tianrui Guan, Divya Kothandaraman, Rohan Chandra, Dinesh Manocha
A learning-based method for identifying safe and navigable regions in off-road terrains and unstructured environments from RGB images. The approach consists of classifying groups of terrain classes based on their navigability levels using coarse-grained semantic segmentation.
arXiv.org
Senthil Hariharan Arul, Dinesh Manocha
The authors present a decentralized collision avoidance method for dense environments that is based on buffered Voronoi cells (BVC) and reciprocal velocity obstacles (RVO). The approach is designed for scenarios with large numbers of close proximity agents and provides passive-friendly collision avoidance guarantees.
2021 IEEE/RSJ International Conference on Intelligent Robots and Systems
Shivang Patel, Senthil Hariharan, Pranav Dhulipala, Ming C Lin1, Dinesh Manocha, Huan Xu, Michael Otte
An important aspect of dynamic urban coverage is how building collision avoidance is incorporated into the overall coverage mission. We consider a multi-agent urban dynamic coverage problem in which a team of flying agents uses downward facing cameras to observe the street-level environment outside of buildings.
GAMMA Lab website; presented at ICRA 2021
Adarsh Jagan Sathyamoorthy, Utsav Patel, Moumita Paul, Nithish K Sanjeev Kumar, Yash Savle, Dinesh Manocha
CoMet is a novel approach for computing a group’s cohesion and using that to improve a robot’s navigation in crowded scenes. This approach uses a novel cohesion-metric that builds on prior work in social psychology.
GAMMA Lab website; presented at IROS 2020
2020
Utsav Patel, Nithish Kumar, Adarsh Jagan Sathyamoorthy, Dinesh Manocha
A novel Deep Reinforcement Learning (DRL) based policy for mobile robot navigation in dynamic environments that computes dynamically feasible and spatially aware robot velocities. The method addresses two primary issues associated with the Dynamic Window Approach (DWA) and DRL-based navigation policies and solves them by using the benefits of one method to fix the issues of the other.
arXiv.org
Senthil Hariharan Arul, Dinesh Manocha
Decentralized collision avoidance algorithms for quadrotor swarms operating under uncertain state estimation. The approach exploits the differential flatness property and feed-forward linearization to approximate the quadrotor dynamics and reciprocal collision avoidance.
arXiv.org
Uttaran Bhattacharya, Christian Roncal, Trisha Mittal, Rohan Chandra, Kyra Kapsaskis, Kurt Gray, Aniket Bera, Dinesh Manocha
Presents an autoencoder-based semi-supervised approach to classify perceived human emotions from walking styles obtained from videos or motion-captured data and represented as sequences of 3D poses.
Computer Vision--ECCV 2020 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X
Yu-Ping Wang, Zi-Xin Zou, Cong Wang, Yue-Jiang Dong, Lei Qiao, Dinesh Manocha
Presents a novel buffering method, ORBBuf, to reduce the impact of data loss on collaborative visual SLAM systems.
arXiv.org
Utsav Patel, Nithish Kumar, Adarsh Jagan Sathyamoorthy and Dinesh Manocha
A novel Deep Reinforcement Learning (DRL) based policy for mobile robot navigation in dynamic environments that computes dynamically feasible and spatially aware robot velocities.
arXiv.org
Qiaoyun Wu, Xiaoxi Gong, Kai Xu, Dinesh Manocha, Jingxuan Dong, Jun Wang
A target-driven navigation system to improve mapless visual navigation for robots in indoor scenes.
arXiv.org
Adarsh Jagan Sathyamoorthy, Utsav Patel, Yash Ajay Savle, Moumita Paul, Dinesh Manocha
Maintaining social distancing norms between humans has become an indispensable precaution to slow dowthe transmission of COVID-19. The paper presents a novel method to automatically detect pairs of humans in a crowded scenario who are not adhering to the 6-foot social distance constraint. The approach makes no assumption about crowd density or pedestrian walking directions. A mobile robot is fitted with commodity sensors, namely an RGB-D camera and a 2-D lidar to perform collision-free navigation in a crowd and estimate the distance between all detected individuals in the camera’s field of view. The robot is also equipped with a thermal camera that wirelessly transmits thermal images to security/healthcare personnel who monitor if any individual exhibits a higher than normal temperature. In indoor scenarios, the mobile robot also can be combined with static mounted CCTV cameras to further improve the performance of the number of social distancing breaches detected. The robot can accurately pursue walking pedestrians etc.
arXiv.org
Tingxiang Fan, Zhiming Che∗, Xuan Zhao, Jing Liang, Cong Shen, Dinesh Manocha, Jia Pan, Wei Zhang
The paper presents a fully autonomous surveillance robot based on a quadruped platform that can promote social distancing in complex urban environments. The researchers mount multiple cameras and a 3D LiDAR on the legged robot. The robot then uses an onboard real-time social distancing detection system to track nearby pedestrian groups. Next, the robot uses a crowd-aware navigation algorithm to move freely in highly dynamic scenarios. The robot finally uses a crowd-aware routing algorithm to effectively promote social distancing by using human-friendly verbal cues to send suggestions to over-crowded pedestrians.
arXiv.org
Jin Liang, Ustav Patel, Adarsh Sathyamoorthy, Dinesh Manocha
A novel high fidelity 3-D simulator that significantly reduces the sim-to-real gap in mobile robotics training collision avoidance policies based on Deep Reinforcement Learning for dense crowd scenarios.
sim2real.github.io
Yuexin Ma, Xinge Zhu, Xinjing Cheng, Ruigang Yang, Jiming Liu, Dinesh Manocha
For intelligent agents like robots and autonomous vehicles, it is crucial to be able to forecast neighboring traffic-agents’ future trajectories for navigation and planning applications. Current methods for trajectory prediction operate in super-vised manners, and therefore require vast quantities of corresponding ground truth data for training. The paper presents a novel, label-free algorithm, AutoTrajectory, for trajectory extraction and prediction to use raw videos directly.
arXiv.org
Adarsh Jagan Sathyamoorthy, Jing Liang, Utsav Patel, Tianrui Guan, Rohan Chandra, Dinesh Manocha
A navigation algorithm for navigating a robot through dense crowds and avoiding collisions by anticipating pedestrian behaviors. The formulation uses visual sensors and a pedestrian trajectory prediction algorithm to track pedestrians in a set of input frames and provide bounding boxes that extrapolate the pedestrian positions in a future time. This hybrid approach combines trajectory prediction with a Deep Reinforcement Learning-based collision avoidance method to train a policy to generate smoother,safer, and more robust trajectories during run-time.
arXiv.org
Liang He, Zherong Pan, Biao Jia, Dinesh Manocha
An algorithm for homogeneous, labeled, and disk-shaped multi-agent motion planning in continuous workspaces with arbitrarily-shaped obstacles.
arXiv.org
Venkatraman Narayanan, Bala Murali Manoghar, Vishnu Sashank Dorbala, Dinesh Manocha, Aniket Bera
ProxEmo is an end-to-end emotion prediction algorithm for socially aware robot navigation among pedestrians. This approach predicts the perceived emotions of a pedestrian from walking gaits, which is then used for emotion-guided navigation taking into account social and proxemic constraints. To classify emotions, ProxEmo uses a multi-view skeletongraph convolution-based model that works on a commodity camera mounted onto a moving robot. The emotion recognition is integrated into a mapless navigation scheme and makes no assumptions about the environment of pedestrian motion.
arXiv.org
Adarsh Jagan Sathyamoorthy, Utsav Patel, Tianrui Guan, Dinesh Manocha
Frozone is a novel algorithm to deal with the Freezing Robot Problem that arises when a robot navigates through dense scenarios and crowds. The method senses and explicitly predicts the trajectories of pedestrians and constructs a Potential Freezing Zone (PFZ); a spatial zone where the robot could freeze or be obtrusive to humans. The formulation computes a deviation velocity to avoid the PFZ, which also accounts for social constraints. Furthermore, Frozone is designed for robots equipped with sensors with a limited sensing range and field of view. The researchers ensure that the robot’s deviation is bounded, thus avoiding sudden angular motion which could lead to the loss of perception data of the surrounding obstacles.
arXiv.org
Tingxiang Fan, Pinxin Long, Wenxi Liu, Jia Pan, Ruigang Yang, Dinesh Manocha
An uncertainty-aware navigation approach for mobile robots that introduces a predictor to model the environmental uncertainty. In addition, the paper proposes an uncertainty-aware navigation network to learn resilient behaviors in prior unknown environments.
2020 International Conference on Robotics and Automation
Jing Liang, Utsav Patel, Adarsh Jagan Sathyamoorthy, Dinesh Manocha
CrowdSteer is a learning-based collision avoidance algorithm for mobile robots operating in dense and crowded environments.
19th International Conference on Autonomous Agents and Multiagent Systems
Jing Liang, Yi-Ling Qiao, Dinesh Manocha
An algorithm that uses commodity visual sensors, including RGB-D cameras and a2D lidar for safe navigation of a mobile robot among pedestrians. The algorithm explicitly predicts the velocities and positions of surrounding obstacles through optical flow estimation and object detection.
arXiv.org
Vrushabh Zinage, Senthil Hariharan Arul, Dinesh Manocha
An online motion planning algorithm for generating smooth, collision-free trajectories for quadrotors operating in an unknown, cluttered 3D environment.
arXiv.org
Aniket Bera, Tanmay Randhavane, Rohan Prinja, Kyra Kapsaskis, Austin Wang, Kurt Gray, Dinesh Manocha
A real-time algorithm that learns the emotion state of the pedestrians to perform socially-aware navigation. The robot learns pedestrians’ emotions and their proxemic constraints to improve both social comfort and navigation.
15th IEEE International Conference on Automatic Face and Gesture Recognition
Jae Sung Park, Dinesh Manocha
A new approach to generate collision-free trajectories for a robot operating in close proximity with a human obstacle in an occluded environment.
arXiv.org
Min Liu, Zherong Pan, Kai Xu, Kanishka Ganguly, Dinesh Manocha
An end-to-end algorithm for training deep neural networks to grasp novel objects.
arXiv.org
Zherong Pan, Xifeng Gao, Dinesh Manocha
A new stress-minimization metric to generate optimal grasps for brittle and fragile objects.
github.io
2019
Qiaoyun Wu, Kai Xu, Jun Wang, Mingliang Xu, Dinesh Manocha
The authors integrate an information-theoretic regularization into a deep reinforcement learning framework for the target-driven task of visual navigation in robotics. This is achieved by first learning to generate a next observation from a current observation and a navigation target, then planning an action toward the target based on the generated observation and the current observation.
arxiv.org
Min Liu, Zherong Pan, Kai Xu, and Dinesh Manocha
Classical robotics research presents the first grasp planning algorithm to compute globally optimal gripper pose that maximizes a grasp metric.
arXiv.org
Senthil Hariharan Arul, Dinesh Manocha
A new algorithm for decentralized collision avoidance for quadrotor swarm navigation in dense environments with static and dynamic obstacles.
arXiv.org
Zhe Hu, Tao Han, Peigen Sun, Jia Pan, Dinesh Manocha
Deep neural network-based controller to servo-control position and shape of deformable objects with unknown deformation properties.
IEEE Robotics and Automation Letters
Qingyang Tan, Tingxiang Fan, Jia Pan, Dinesh Manocha
A novel algorithm (DeepMNavigate) for global multi-agent navigation in dense scenarios using deep reinforcement learning. | Watch a video about DeepMNavigate |
qytan.com
Rohan Chandra, Uttaran Bhattacharya, Trisha Mittal, Xiaoyu Li, Aniket Bera, Dinesh Manocha
The GraphRQI algorithm identifies driver behaviors from road agent trajectories. It is 25 percent more accurate over prior behavior classification algorithms for autonomous vehicles. | Watch a video about GraphRQI |
arxiv.org
Qingyang Tan, Zherong Pan, Lin Gao, and Dinesh Manocha
A new method bridges the gap between mesh embedding and physical simulation for efficient dynamic models of clothes. The key technique is a graph-based convolutional neural network (CNN) defined on meshes with arbitrary topologies, and a new mesh embedding approach based on physics-inspired loss term. After training, the learned simulator runs10–100 times faster and the accuracy is high enough for robot manipulation tasks. | Watch a video about this method |
arxiv.org
2023
Mohamed Khalid Jaffar, Michael Otte
The paper proposes an online single-query sampling-based motion re-planning algorithm using finite-time invariant sets, commonly referred to as “funnels.” The authors combine concepts from nonlinear systems analysis, sampling-based techniques, and graph-search methods to create a single framework that enables feedback motion re-planning for any general nonlinear dynamical system in dynamic workspaces.
International Journal of Robotics Research
George Kontoudis, Michael Otte
Active Learning of Gaussian process (GP) surrogates is an efficient way to model unknown environments in various applications. In this paper, we propose an adaptive exploration-exploitation active learning method (ALX) that can be executed rapidly to facilitate real-time decision making. For the exploration phase, we formulate an acquisition function that maximizes the approximated, expected Fisher information. For the exploitation phase, we employ a closed-form acquisition function that maximizes the total expected variance reduction of the search space. The determination of each phase is established with an exploration condition that measures the predictive accuracy of GP surrogates. Extensive numerical experiments in multiple input spaces validate the efficiency of our method.
preprint on Kontoudis website
Sharan Nayak;, Michael Paton, Michael Otte
The paper presents a heuristic-guided multi-robot motion planning framework that solves the problem of n dynamical agents visiting m unlabeled targets in a partially known environment for planetary surface missions without solving the two-point boundary value problem.
2023 IEEE Robotics and Automation Letters
Jae-Kyung Cho, Chan Kim, Mohamed Khalid M Jaffa, Michael Otte, Seong-Woo Kim
The paper focuses on training a low-level controller, which operates in response to dynamics-changes without prior knowledge or fine-tuning of the parameters, using reinforcement learning. The authors randomize the dynamics of quadrotors in the simulator and train the policy based on dynamics information extracted from the state-action history through recurrent neural networks (RNNs). They propose a novel structure with excellent performance, verified by testing experiments that stabilize quadrotors with different dynamics.
2023 IEEE International Conference on Robotics and Automation (ICRA)
Alkesh Srivastava, George Kontoudis, Donald Sofge, Michael Otte
In some missions it is typical for target elements to be in close proximity to hazardous factors that hinder the information-gathering process. In this study, we examine a similar scenario and conduct experiments to determine if additional knowledge about the correlation between hazards and targets improves the efficiency of information gathering. To incorporate this knowledge, the researchers utilize a Bayesian network representation of domain knowledge and develop an algorithm based on this representation. An empirical investigation reveals that such additional information on correlation is beneficial only in environments with moderate hazard lethality, suggesting that while knowledge of correlation helps, further research and development is necessary for optimal outcomes.
arXiv.org
George Kontoudis, Michael Otte
The authors propose an active learning method of expected variance reduction with Gaussian process (GP) surrogates using a closed-form gradient. The use of closed-form gradient leads the optimization to find better solutions with reduced computations. They derive the closed-form gradient for active learning Cohn (ALC) using GP surrogates that are formed with the separable squared exponential covariance function.
researchgate.net
Ilya Semenov, Robert Brown, Michael Otte
A dynamic path planning algorithm to navigate an amphibious rotor craft through a concave time-invariant obstacle field while attempting to minimize energy usage.
arXiv.org
2022
Mohamed Jaffar, Michael Otte
The authors introduce an online single-query sampling-based feedback motion re-planning algorithm using finite-time invariant sets, “funnels". They combine concepts from nonlinear systems analysis, sampling-based motion planning, and graph-search methods to create a single framework that enables feedback motion planning/replanning for general nonlinear dynamical systems in a dynamic workspace. They introduce a novel graph data structure to represent a network of volumetric funnels, enabling the use of quick graph-replanning techniques.
WAFR 2022: Algorithmic Foundations of Robotics XV
Alkesh Srivastava, George Kontoudis, Donald Sofge, Michael Otte
A distributed information-gathering algorithm for multi-robot systems that use multiple path-based sensors to infer the locations of hazards within the environment.
researchgate.net
Mitali Gandhe, Michael Otte
The authors compare the resilience of four distributed robot swarm clustering algorithms to masquerade attacks launched from malicious robots within the swarm.
15th International Workshop on the Algorithmic Foundations of Robotics (WAFR 2022)
2023
Wei Cui, Animesh Shastry, Derek Paley, Stephen Nogar
An autonomous search and revisit strategy for multi-agent unmanned aerial systems (UAS), specifically tailored for long-range operations where communication may be limited or denied.
CDCL paper
Artur Wolek, Derek Paley
An estimation and control framework to stabilize the parallel motion of a school of robotic fish using sensory feedback. Each robot is modeled as a constant speed, planar, self-propelled particle that produces a flowfield according to a dipole potential flow model. An artificial lateral line system senses pressure fluctuations at several locations along each robot’s body. The equations of motion and measurement model are formulated in a path frame that translates and rotates with each robot’s position and velocity, respectively. A particle filter estimates the relative position and heading of other nearby robots. The resulting estimate drives a Lyapunov-based formation controller to synchronize the headings of the robots and achieve a parallel formation. Numerical simulations illustrate the proposed approach.
CDCL paper; IROS 2023
Anthony Thompson, Ashley Peterson, Matthew McHenry, Derek Paley
This paper investigates a pursuit-evasion game with a single pursuer and evader in a bounded environment, inspired by observations of predation attempts by lionfish (Pterois sp.). The pursuer tracks the evader with a pure pursuit strategy while using an additional bioinspired tactic to trap the evader, i.e. minimize the evader's escape routes. Specifically, the pursuer employs symmetric appendages inspired by the large pectoral fins of lionfish, but this expansion increases its drag and therefore its work to capture the evader. The evader employs a bioinspired randomly-directed escape strategy to avoid capture and collisions with the boundary. The authors investigate the trade-off between minimizing the work to capture the evader and minimizing the evader's escape routes. By using the pursuer's expected work to capture as a cost function, they determine when the pursuer should expand its appendages as a function of the relative distance to the evader and the evader's proximity to the boundary. Visualizing the pursuer's expected work to capture everywhere in the bounded domain, yields additional insights about optimal pursuit trajectories and illustrates the role of the boundary in predator-prey interactions.
Bioinspiration & Biomimetics
Debdipta Goswami, Ari Riggins, Derek Paley
E-scooters are the latest development for urban micromobility solutions. Due to their small sizes and effectiveness as a first-/last-mile solution, dockless e-scooters are increasingly popular in U.S. cities. The lack of widespread statistical studies of e-scooter mobility patterns forces many e-scooter rental companies to resort to empirical methods for deploying the vehicles. This article proposes a data-driven method to predict e-scooter mobility in urban areas. Scooter trip origins and destinations are partitioned by geographical region, and the trip origin volume is predicted as a time series using an echo-state network coupled with an ensemble Kalman filter. A Markov operator prediction technique is used to predict the spatiotemporal distribution of the scooter trip destinations.
IEEE Control Systems Magazine
2022
Cong Wei, Derek Paley
This paper formulates a strategy for Driftcam, an ocean-going robot system, to observe and track the motion of an ocean biological phenomenon called the pelagic scattering layer, which consists of organisms that migrate vertically in the water column once per day. Driftcam’s horizontal motion is determined by the flow field and the vertical motion is regulated by onboard buoyancy control. To observe the evolution of the scattering layer, an ensemble Kalman filter is applied to estimate organism density; the density dynamics are propagated using the Perron-Frobenius operator. Multiple Driftcam are subject to depth regulation by open-loop and closed-loop controllers; a control strategy is proposed to track the peak of the density. Numerical simulations illustrate the efficacy of this strategy and motivate ongoing and future efforts to design a coordination formation algorithm for multi-agent Driftcam system to track the motion of the scattering layer, with implications for ocean monitoring.
CDCL paper
Alexander Donkels, Johann Dauer, Derek Paley
An analysis of the dynamics and stability of a nonlinear model of a small-scale tethered rotorcraft. The authors develop a simplified model of a rotorcraft’s longitudinal dynamics and vary model parameters including tether force, trim conditions, and the horizontal wind to study the interdependence of those parameters and their impact on the model’s equilibrium points and stability. This is a preliminary step towards design of an automatic control of an unmanned rotorcraft capable of autonomous tethered landing and development of tether force control laws for the winch device.
CDCL paper
Cong Wei, Derek Paley
This paper presents a distributed coordination algorithm for multiple, buoyancy controlled underwater robots to achieve a moving formation in a shear flow.
CDCL paper
2021
Sheng Cheng, Derek Paley
This paper describes an optimization framework to control a distributed parameter system (DPS) using a team of mobile actuators. The framework simultaneously seeks optimal control of the DPS and optimal guidance of the mobile actuators such that a cost function associated with both the DPS and the mobile actuators is minimized subject to the dynamics of each.
Automatica
Sheng Cheng, Derek Paley
Describes an optimization framework to design guidance for a possibly heterogeneous team of multiple mobile sensors to estimate a spatiotemporal process modeled by a 2D diffusion-advection process.
arXiv.org—submitted to Automatica
Lena Johnson, Derek Paley, Hugh Bruck
A hybrid state-space model of a UAV capable of flapping and gliding.
AIAA Journal of Guidance, Control and Dynamics
Eric Berkenpas, Charles Shepard, Rachel Suitor, Paul Zaidins, Derek Paley, Kyle Abernathy
A paper about the design, development, and field results of novel depth-controlled sensor platforms. Called the “Swarming Driftcams,” these platforms travel passively in horizontal direction with ocean currents while controlling and profiling depth. All aspects of the vehicle hardware are discussed.
CDCL paper
John Lathrop, Derek Paley
Presents a state-space model of worm-inspired burrowing locomotion using discrete elastic rod theory applied to a segmented body representing the worm’s body. The model is controlled by changing the intrinsic length of each segment, with several peristaltic travelling-wave gaits considered. Simulations varying gait parameters show that some travelling-wave gaits allow the worm model to make faster progress. The derived state-space model permits improved control and estimation for worm-inspired robots and empirically shows how differing travelling-wave gaits affect the speed of locomotion.
CDCL paper
Sheng Cheng, Derek Paley
This paper describes an optimization framework to control a distributed parameter system using a team of mobile actuators.
Paper submitted to Automatica
Erin Fowler, Stella Hurtt, and Derek Paley
A dynamic simulation of the cislunar environment for use in numerical analysis of various pairings of resident space objects and sensing satellites intended for cislunar space domain awareness.
CDCL paper
Lena Johnson, Derek Paley, Hugh Bruck
A hybrid flapping-gliding dynamic model for a flapping-wing unmanned aerial vehicle. The model combines the nonlinear phugoid dynamics of gliding flight, the equations of motion of the onboard servo motors, and an equivalent circuit discharge model for the onboard battery.
AIAA Scitech 2021 Forum
2020
Sheng Cheng, Derek Paley
A framework to design guidance for a team of mobile sensors to estimate a distributed parameter system modeled by a diffusion process.
CDCL paper
William Scott, Prateek Jaya Prakash, Derek A. Paley
Many organisms achieve locomotion through undulatory traveling-wave motions, both on land and underwater. Recently, caterpillar locomotion has been modeled using the theory of planar discrete elastic rods (PDER) and open-loop control. This work considers a planar discrete elastic rod model with closed-loop control over the intrinsic material parameters of length and curvature distributed along its length. The authors introduce local curvature feedback control laws to drive the shape of the robot to a desired traveling-wave reference trajectory, where the phase of the wave is determined via a central pattern generator or via distributed control with a circulant communication topology. Through numerical simulations utilizing simple models of fluid–body interaction forces we examine undulatory eel-like swimming gaits that give rise to net forward motion. These results show promise for the design of distributed feedback control laws in modular soft robotic systems.
Chapter in Bioinspired Sensing, Actuation, and Control in Underwater Soft Robotic Systems (Springer 2021)
Jordan Boehm, Eric Berkenpas, Charles Shepard, Derek A. Paley
The paper compares output feedback control strategies for an underwater thruster.
IEEE Journal of Oceanic Engineering
Brian Free, Jinseong Lee, Derek Paley
The paper presents a state-feedback control law for steering a fish-inspired robot in a desired direction, and engaging it in pure pursuit.
Bioinspiration and Biomimetics
Artur Wolek, Sheng Cheng, Debdipta Goswami, Derek Paley
A cooperative mapping and target-search algorithm for a team of autonomous quadrotors equipped with noisy, range-limited sensors. The algorithm can concurrently map and search an unknown urban area, while detecting and tracking a mobile ground target.
IEEE Robotics and Automation Letters
Sheng Cheng, Derek Paley
An optimization framework to control a distributed parameter system (DPS) using a team of mobile actuators. The optimization simultaneously seeks efficient guidance of the mobile actuators and effective control of the DPS such that an integrated cost function associated with both the mobile actuators and the DPS is minimized.
CDCL paper
Travis Burch, John Lathrop, William Scott, Derek Paley
A state-space description using planar discrete elastic rod theory of a soft robotic appendage with torque input at one end. The authors design a linear output feedback controller to balance the appendage in an unstable vertical configuration with an angle sensor and torque input co-located at the base.
CDCL paper
William Craig, Derrick Yeo, Derek Paley
The researchers use a model of the aerodynamic interaction between the propellers and wind, paired with onboard flow sensing and feedback control, to improve the stability of quadrotors in unsteady winds with the long-term goal of enabling reliable outdoor flight in windy conditions.
AIAA Journal of Guidance, Control, and Dynamics
Erin Fowler, Stella Hurtt, Derek Paley
Quantitative assessments of the orbits and sensor characteristics of satellites intended for cislunar space domain awareness.
2nd IAA Conference on Space Situational Awareness
2019
Justin M. Lidard, Debdipta Goswami, David Snyder, Girguis Sedky, Anya Jones, Derek Paley
Unsteady aerodynamics is driving research at the interface of fluid dynamics and control theory for low Reynolds number aircraft such as micro air vehicles (MAVs). The regulation and control of unsteady behavior is crucial for maintaining the stability of an MAV, which necessitates accurate modeling of their flight surfaces. This paper describes the implementation of the Goman-Khrabrov model for flow stagnation near an actuated airfoil with a feedback-controlled pitch rate for the purpose of maximizing the time-averaged unsteady lift.
AIAA SciTech 2020 Forum
Francis D. Lagor, Kayo Ide, Derek Paley
An architecture for estimation of a flow field using a hypothetical oceanographic vehicle that is guided along paths of high flow-field observability, a concept quantifying the informativeness of a path.
IEEE Journal of Oceanic Engineering
Derek Paley, Artur Wolek
A review of control of mobile sensor networks for environmental monitoring and other applications. Includes sensor platform dynamics and cooperative control and estimation; process modeling and estimation for both continuous and discrete models; sampling metrics and optimization, including coverage-, topology-, information-, and estimation-based metrics; and methods for task design and allocation.
Annual Review of Control, Robotics, and Autonomous Systems
William Craig, J. T. Lewis, Derek Paley
This paper describes a linear controller that has been optimized for gust rejection using CONDUITR©, based on system identification performed with CIFERR©. Additionally, a custom flow probe package was used to investigate the benefits of flow feedback for gust rejection. Experiments were performed with a 210mm quadrotor system running Cleanflight software, where flow feedback yielded improvements for both short and long time-scale gusts, particularly for the longer time-scale five second gusts when the vehicle had time to settle in the wind.
VFS Autonomous VTOL Technical Meeting and Electric VTOL Symposium
2019
Ahmed Ramadan, Anindo Roy, Elisabeth Smela
This paper provides proof of concept for a novel method to recognize impaired push-off and foot-drop deficits in hemiparetic gait using analog pressure sensors. These data may enhance feedback from a modular ankle exoskeleton (such as Anklebot) for stroke rehabilitation.
IEEE Journal of Translational Engineering in Health and Medicine
2023
Noah Barnes, Olivia Young, Adira Colton, Xiaolong Liu, Miroslaw Janowski, Dheeraj Gandhi, Ryan Sochol, Jeremy Brown, Axel Krieger
During minimally invasive surgery, surgeons maneuver tools through complex anatomies, which is difficult without the ability to control the position of the tools inside the body. A potential solution for a substantial portion of these procedures is the efficient design and control of a pneumatically actuated soft robot system. The authors designed and evaluated a system to control a steerable catheter tip. A macroscale 3D printed catheter tip was designed to have two separately pressurized channels to induce bending in two directions. A motorized hand controller was developed to allow users to control the bending angle while manually inserting the steerable tip. Preliminary characterization of two catheter tip prototypes was performed and used to map desired angle inputs into pressure commands. The integrated robotic system allowed both a novice and a skilled surgeon to position the steerable catheter tip at the location of cylindrical targets with sub-millimeter accuracy. The novice was able to reach each target within ten seconds and the skilled surgeon within five seconds on average.
International Journal of Computer Assisted Radiology and Surgery
2023
Michael Bonthron, Eleonora Tubaldi
Bistable shallow arches are ubiquitous in many engineering systems ranging from compliant mechanisms and biomedical stents to energy harvesters and passive fluidic controllers. In all these scenarios, the bistable states of the arch and the sudden transition between them via snap-through instability are harnessed. However, bistable arches have been traditionally studied and characterized by triggering snap-through instability using quasi-static forces. Here, the authors analytically examine the effect of oscillatory loads and investigate the dynamic behavior of bistable arches spanning from intrawell motion to periodic and chaotic interwell motion.
ASME Journal of Applied Mechanics
Snehi Shrestha, Kieran James Barvenik, Tianle Chen, Haochen Yang, Yang Li, Meera Muthachi Kesavan, Joshua M. Little, Eleonora Tubaldi, Po-Yen Chen
An integrated workflow is developed to combine collaborative robotics with machine learning to accelerate the design of conductive aerogels with programmable electrical and mechanical properties. The model-suggested conductive aerogels with high electrical conductivity, customized compression resilience, and pressure insensitivity allow for compression-stable Joule heating for wearable thermal management. The fusion of robotics-accelerated experimentation, machine intelligence, and simulation tools expedites the tailored design and scalable production of conductive aerogels.
Research Square
Kieran J Barvenik, Zachary Coogan, Matteo Pezzulla, Eleonora Tubaldi
Shell buckling presents numerous opportunities to program fast and reversible reconfigurations into multifunctional devices. Applications such as soft actuators, locomotion systems, and biomedical instruments can leverage the diverse deformation properties of shell buckling mode shapes. Here, we investigate the buckling behavior of soft, thin-shell domes with free boundary conditions to facilitate a novel gripper design. Tuning the buckling mode and pressure could enhance the gripper's actuation speed and energy input characteristics. We will present analytical and numerical models used to find the dome's buckling pressure as a function of its material and geometric properties, such as its slenderness ratio and spherical cap angle. We analyze the shell's quasi-static response during pressurization with finite element methods (FEM), and we discuss good agreement between the two models. For experimental validation, we investigate a hemispherical soft shell with a thin film covering the leading edge to create an isolated fluid cavity. Using similar FEM, we model the fluid cavity to compare to the experimental results. Our research aims to advance knowledge about the buckling of soft, thin-shell domes and to inform the usage of hemispherical shells as soft grippers. This investigation also has the potential to pave a path forward for future research into soft devices with buckling controlled actuation.
American Physical Society March Meeting 2023
Michael Bonthron, Eleonora Tubaldi
Multistable mechanical metamaterials have recently provided unique platforms with wave guidance and wave steering capabilities. Arrays of symmetric/asymmetric bistable elements, such as elastically/plastically deformed arches, can be patterned to control the signal transmission and to passively achieve nonreciprocal wave propagation. In this work, sequential nonlinear transition wavefronts will be obtained by imposing an oscillatory transverse displacement to an element of a multistable metamaterial made of one-dimensional array of arches. Different excitation amplitude and frequencies will be considered and it will be showed how these features can be harnessed to control transition wavefronts propagation. In addition, the effect of multiple oscillatory inputs at different positions of the array will be investigated. The dynamics of the system will be theoretically studied by solving the continuum equations of motion with both direct time integration methods and numerical continuation techniques (i.e., pseudo-arclength continuation). Nonlinear wave interference, collision-like cases, and internal resonance phenomena will be characterized based on different oscillatory excitation parameters. This study will open avenues to the use of dynamic inputs to control sequential transition wavefronts in mechanical metamaterials.
American Physical Society March Meeting 2023
Anthony Jones, Eleonora Tubaldi
In the field of soft robotics, the nonlinearities and large deformations typical of elastomeric materials are used to emulate complex phenomena typical of the animal kingdom. Introducing metamaterials further expands the design space of soft robotics, unlocking interesting and useful mechanical properties and granting novel functionalities to components. Here, we study a soft metamaterial that undergoes a global pattern reconfiguration due to buckling instability upon inflation. Studies thus far have induced global buckling instability through uniaxial compression or negative pressure, but we will introduce and leverage, for the first time, a global buckling instability due to positive pressure in a porous metamaterial. Both numerical and experimental approaches will be used to obtain pressure-volume curves and to characterize the buckling modes based on the geometrical properties of the 3D porous metamaterial. We will show how the buckling-induced reconfigurable and reversible behavior can be harnessed for gripping and sequential actuation purposes. Leveraging this instability as a means for actuation, we explore a new class of programmable soft actuators with distributed gripping and embedded sequencing capabilities.
American Physical Society March Meeting 2023
2023
S. Gart, M. Bundy, R. Vonwahlde, K. Jayarajah, K. Rippy, F. Ayoub, J. Chen, H. Zhuang, Z. Nzeako, S. Muntaha, K. Nwaiwu, W. Mackinnon, S. Mukku, D. Aderinwale, B. Lin, J. Scheffler, D. Zhu, M. Zheng, S. Roy, J. Mirenzi, B. Amaya, D. Hu, Huan Xu, Derek Paley
The MRC-ARL Summer Student Team Research Experience, a joint effort of the Maryland Robotics Center (MRC) and the US Army Combat Capabilities Development Command (DEVCOM) Army Research Laboratory (ARL), engaged 20 undergraduate students from the University of Maryland, Baltimore County and the University of Maryland, College Park. Their mission centered on the conception and construction of a system for the airdrop deployment of a ground-based robot. This was motivated by the quest for safer, more efficient alternatives to current methods of delivering critical supplies and personnel in high-risk scenarios, reducing both danger and resource intensiveness. Over the course of an 8-week program, the students focused on creating an impact energy mitigation system and developing algorithms to facilitate ground navigation post-impact. Their collective efforts culminated in a final, real-world test scenario at ARLs Robotics Research Collaboration Campus. Here, robots were dropped from a 12-m building. Although the students demonstrated solutions were only partially successful, they learned valuable lessons on teamwork and project management and set the stage for future advances in air-dropped robotic systems.
Defense Technical Information Center
Donald Costello, Nicholas Hanlon, Huan Xu
Pilots use their senses and training to generate situational awareness (SA). They then use this SA to make sound aeronautical decisions. Autonomous vehicles, by contrast, cannot rely on pilot expertise in off-nominal situations. They must rely on their onboard sensors to build SA of the environment. As these sensors degrade, it is hypothesized that a point exists where the SA generated by these sensors is inadequate to allow the autonomous vehicle to make sound aeronautical decisions. In previous work, a point was defined based on broad assumptions within a modeling and simulation environment (i.e., the error within each sensor was known and not random). This research used a larger data set that contained random errors within the sensors. The data was then used to build predictive equations through a Monte Carlo simulation in the same simulation environment as previous work. While the data showed there was a statistically significant relationship between the error values in each sensor and the fused distance value, the resulting predictive equations were not able to provide adequate SA to make sound aeronautical decisions. This research highlights multiple issues the test and evaluation community will face when trying to develop new techniques for the verification and validation of autonomous systems.
Systems Engineering (INCOSE)
Donald Costello, Huan Xu
The United States Navy plans on fielding autonomous surface vessels in the near future. This paper presents a preliminary approach for certifying an autonomous surface vessel to complete a transoceanic voyage. It decomposes the rules currently outlined by the Convention on the International Regulations for Preventing Collisions at Sea for avoiding a collision when two vessels are in sight of one another in international waters.
Maritime Policy & Management
Akshay Bapat, Bharath Reddy Bora, Jeffrey Herrmann, Shapour Azarm, Huan Xu, Michael Otte
Explores the problem of task allocation among networked multi-robot systems when communication conditions are poor. The authors develop solutions that ensure all tasks are eventually completed—even if some agents in the network are destroyed. Their two new task allocation algorithms assume communication may not happen, but benefit the robotic agents and their missions whenever communication is successful.
IEEE Access
Donald Costello, Huan Xu
The methods and procedures within United States naval aviation to certify an aircraft safe for flight are well established. However, these methods and procedures are based on clearing a system that is operated or monitored by a human. A fully autonomous system will not have a human in or on the loop and will therefore require a new method for certifying it safe for flight. This paper details how to use run time assurance as the framework for a safety of flight certification of autonomous behavior within United States naval aviation.
Systems Engineering, an INCOSE journal
2022
Huan Mumu Xu, Joseph Borson
Autonomous systems hold promise for significant improvements in fields from aircraft to cars to medical devices. These systems will be subject to legal governance requirements. While some regulatory principles apply in general to autonomous systems, an ‘‘autonomy-focused’’ approach is an inadequate lens to consider their governance. A better approach takes into consideration that autonomous systems are improvements of currently regulated complex systems, and that the regulation of autonomous elements will occur within those systems’ preexisting regulatory frameworks. In this work, the authors describe what such a framework might look like.
IEEE Access
2021
Shivang Patel, Senthil Hariharan, Pranav Dhulipala, Ming C Lin1, Dinesh Manocha, Huan Xu, Michael Otte
An important aspect of dynamic urban coverage is how building collision avoidance is incorporated into the overall coverage mission. We consider a multi-agent urban dynamic coverage problem in which a team of flying agents uses downward facing cameras to observe the street-level environment outside of buildings.
GAMMA Lab website; presented at ICRA 2021
2020
Donald H. Costello III, Jason Jewell, Huan Xu
The current safety of flight clearances for unmanned aircraft requires a qualified operator who can make decisions and ultimately bears the responsibility for the safe operations of the vehicle. The future of aviation is unmanned, and ultimately autonomous. Yet, a method for certifying an autonomous vehicle to make decisions currently reserved for qualified pilots does not exist. Before we can field autonomous systems, a process needs to be approved to certify them.
This paper analyzes the flight-test data (both developmental and operational) of an autonomous decision engine selecting an appropriate landing site for a large rotorcraft in an unprepared landing zone. In particular, this paper focuses on using legacy test and evaluation methods to determine their suitability for obtaining a safety of flight clearance for a system that possesses autonomous functionality.
The autonomous system under test was able to complete a mission currently reserved for qualified pilots under controlled conditions. However, when confronted with conditions that were not anticipated (or programmed), the software lacked the judgment a pilot uses to complete a mission under off-nominal conditions.
AIAA Journal of Air Transportation
Andrew Poissant, Lina Castano, Huan Xu
The researchers have developed a UAV hazard mitigation software that guards against ground impact in highly populated areas. Successful incorporation of the ground impact and hazard mitigation (GIHM) module into UAV software reduces fatalities per flight hour and brings UAVs closer to being safe enough for integration into the NAS.
AIAA Journal of Aerospace Information Systems
Shivang Patel, Senthil Hariharan, Pranav Dhulipala, Ming C Lin, Dinesh Manocha, Huan Xu, Michael Otte
A study of multi-agent coverage algorithms for autonomous monitoring and patrol in urban environments. The researchers consider scenarios in which a team of flying agents uses downward facing cameras (or similar sensors) to observe the environment outside of buildings at street-level. They conduct an empirical evaluation of six multi-agent coverage algorithms in urban environments.
arXiv.org
Ruchir Patel, Eliot Rudnick-Cohen, Shapour Azarm, Michael Otte, Huan Xu, Jeffrey W. Herrmann
In multi-agent collaborative search missions, task allocation is required to determine which agents will perform which tasks. The paper proposes a new approach for decentralized task allocation based on a decentralized genetic algorithm.
2020 International Conference on Robotics and Automation
Sharan Nayak, Mohamed Khalid M. Jaffar, Estefany Carrillo, Suyash Yeotikar, Eliot Rudnick-Cohen, Ruchir Patel, Jeffrey Herrmann, Huan Xu, Shapour Azarm, Michael Otte
An experimental comparison of the performance of five state-of-the-art decentralized task allocation algorithms under imperfect communication conditions for teams of unmanned aerial vehicles (UAVs).
IEEE Robotics and Automation Letters
2022
Xiaomin Lin, Nare Karapetyan, Kaustubh Joshi, Tianchen Liu, Nikhil Chopra, Miao Yu, Pratap Tokekar, Yiannis Aloimonos
Autonomous navigation in the underwater environment is challenging due to limited visibility, dynamic changes, and the lack of a cost-efficient accurate localization system. We introduce UIVNAV, a novel end-to-end underwater navigation solution designed to drive robots over Objects of Interest (OOI) while avoiding obstacles, without relying on localization. UIVNAV uses imitation learning and is inspired by the navigation strategies used by human divers who do not rely on localization.
arXiv.org
Gurtajbir Singh Herr, Lasitha Weerakoon, Miao Yu, Nikhil Chopra
The authors utilize an optimization based planner, namely TEB local planner, as the expert to generate high quality motion commands for a car-like robot operating in a simulated dynamic environment. They use these labels to train a deep neural network that learns to navigate. The deep learning-based planner is further augmented with safety margins to enhance its effectiveness in collision avoidance.
Proceedings of the ASME 2022 International Mechanical Engineering Congress and Exposition IMECE2022
Tianchen Liu, Miao Yu, Nikhil Chopra
Navigation of autonomous underwater vehicles (AUVs) is critical for many vital tasks operated in the aquatic environment. However, the navigation problem is challenging in confined environments like caves. Since humans can usually solve such a problem with minimal effort, the main objective of this paper is to implement learning-based neural network models for AUVs to learn from human actions in a confined environment. The Girona underwater dataset, acquired in a cave exploration experiment, was used for the learning task. The odometry and sonar data from the dataset were used to train two deep neural network models for AUV navigation through the underwater caves. The test data demonstrates that both models achieve relatively high accuracy for the navigation task.
IEEE OCEANS 2022 Conference, Hampton Roads, Va.
Lasitha Weerakoon, Gurtajbir Singh Herr, Jasmine Blunt, Miao Yu, Nikhil Chopra
The authors study algorithms for detecting and including glass objects in an optimization-based Simultaneous Localization and Mapping (SLAM) algorithm for safe and reliable robot navigation. They utilize a simple and computationally inexpensive glass detection scheme for detecting glass objects and present the methodology to incorporate the identified objects into the occupancy grid maintained by such an algorithm (Google Cartographer).
arXiv.org
Xiaomin Lin, Cheng Liu, Allen Pattillo, Miao Yu, Yiannis Aloimonos
SeaDroneSim is a new benchmark suite that can be used to create photo-realistic aerial image datasets with ground truth for segmentation masks of any given object.
Computer Vision Foundation 2023 WACV Workshop
2021
Behzad Sadrfaridpour, Yiannis Aloimonos, Miao Yu, Yang Tao, Donald Webster
To test the idea that advancements in robotics and artificial intelligence offer the potential to improve the monitoring of oyster beds, the researchers prepared a remote operated underwater vehicle (ROV) with a camera and filmed in the Chesapeake Bay. They then used these videos to train convolutional neural networks (CNNs) to count oysters and track them in consecutive image frames so they are not identified multiple times.
arXiv.org
2020
Keshav Rajasekaran, Hyung Dae Bae, Sarah Bergbreiter, Miao Yu
A novel method to fabricate a micro-scale artificial hair sensor capable of directional airflow sensing.
2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
