In his novel "A Deepness in the Sky" Vernor Vinge offered a compelling vision of a future where armies of small, lightweight robotic elements would pervade an environment. Such a system would allow a user to automatically gather and analyze data from every corner of the space, to manipulate remote objects, to communicate with other users of the swarm and to carry out distributed computations. In the engineering community, the idea of deploying teams of small inexpensive robotic agents to accomplish various tasks is one that has gained increasing currency over the last few years.

This paradigm offers several compelling advantages. Multiple robots can be distributed around objects being manipulated to provide mechanical advantages and simplify planning problems. Sensor information gathered from multiple viewpoints simultaneously can be integrated allowing the system to form a more complete and accurate understanding of the state of the environment. The team concept also offers a certain amount of robustness since the failure of any one robot can be compensated for by the actions of the other team members. Additionally, since the robots are spatially distributed, it is less likely that any single catastrophe will completely destroy the capabilities of the ensemble. At the University of Pennsylvania we have assembled a multidisciplinary team of investigators from three departments (Computer and Information Science, Electrical Engineering and Mechanical Engineering) who are conducting research into several of the problems raised by this paradigm of distributed robotic agents. Our efforts in this area are divided into three main thrusts:

• Coordinated Motion Control: The first thrust deals with the problems associated with coordinating the motion of teams of robots. Some of the questions that are addressed by this effort include the problem of controlling the motion of robots moving in formation and coordinating the action of robots engaged in cooperative manipulation of an object.
• Cooperative Sensing: The second thrust focuses on the issues associated with combining the information obtained from distributed robots to form a coherent model of the environment. One of the interesting opportunities afforded by considering distributed teams of robots is the option of dynamically moving the robots in order to improve the estimates derived from the sensors.
• Mobile Networking: The third area of research concerns the problems associated with designing and analyzing networking strategies that are appropriate for use with distributed teams of robotic agents. Since the platforms are mobile, many of the traditional networking strategies, which were designed with fixed infrastructure in mind, are not applicable. As part of this proposal we intend to investigate questions concerning the appropriateness of various wireless networking technologies such as IEEE 802.11b and Bluetooth.

It is important to note that our research agenda is germane not only to the field of robotics but to computer science in general. With the advent of the Internet it is increasingly the case that we are surrounded by a sea of sensors, actuators and computational elements connected by networks. The techniques developed in the context of our research would also be relevant to the problems associated with marshalling these distributed resources to carry out useful tasks.

Many, though not all, of our experiments to date have been based on the ClodBuster platform shown below. With these platforms we have demonstrated, cooperative localization, formation control, coordinated manipulation, distributed mapping, online sensor planning and a number of other applications.

This project involved developing the software required to guide a tracked mobile platform up a staircase.We adopted a Gibsonian approach to the problem and developed a simple scheme that analyzed patches in the images to decide on the orientation of the vehicle with respect to the staircase. The entire scheme was implemented in real time on a 500MHz Pentium processor.
 

The Ben Franklin Racing Team’s goal is to build fast, reliable, safe and autonomous vehicles that will revolutionize transportation systems in urban environments. We will leverage state-of-the-art advances in sensing, control theory, machine learning, automotive technology and artificial advantages to build robotic cars. The team will participate the 2007 DARPA Urban Challenge.

This multi-university project involves the University of Pennsylvania, the University of California at Berkeley, and Carnegie Mellon University. It focuses on the design and evaluation of the adaptive hierarchical control of mixed autonomous and human operated semi-autonomous teams that deliver high levels of mission reliability despite uncertainty arising from rapidly evolving environments and malicious interference from an intelligent adversary. Equipment for this project is supported by an ARO DURIP grant.

The SWARMS project brings together experts in artificial intelligence, control theory, robotics, systems engineering and biology with the goal of understanding swarming behaviors in nature and applications of biologically-inspired models of swarm behaviors to large networked groups of autonomous vehicles. Our main goal is to develop a framework and methodology for the analysis of swarming behavior in biology and the synthesis of bio-inspired swarming behavior for engineered systems. We will be interested in such questions as: Can large numbers of autonomously functioning vehicles be reliably deployed in the form of a “swarm” to carry out a prescribed mission and to respond as a group to high-level management commands? Can such a group successfully function in a potentially hostile environment, without a designated leader, with limited communications between its members, and/or with different and potentially dynamically changing “roles” for its members? What can we learn about how to organize these teams from biological groupings such as insect swarms, bird flocks, and fish schools? Is there a hierarchy of “compatible” models appropriate to swarming/schooling/flocking which is rich enough to explain these behaviors at various “resolutions” ranging from aggregate characterizations of emergent behavior to detailed descriptions which model individual vehicle dynamics?

This project is investigating and developing methods for the recovery of 3D underground structures from subsurface non-invasive measurements obtained with ground penetrating radar, magnetometry, and conductivity sensors. The results will not only provide hints for further excavation but also 3D models that can be studied as if they were already excavated. The three fundamental challenges investigated are the inverse problem of recovering the volumetric material distribution, the segmentation of the underground volumes, and the reconstruction of the surfaces that comprise interesting structures.

The goal of the LAGR program is to develop a new generation of learned perception and control algorithms for autonomous ground vehicles, and to integrate these learned algorithms with a highly capable robotic ground vehicle.

This project, in collaboration with the University of Minnesota and the California Institute of Technology, addresses research issues key to an important application of robot teams and information technology (emergency response in hazardous environments for various tasks). The research focuses on the development of methods for team coordination and dynamic distribution of tasks to robots. The project integrates the algorithms with first responder teams, emphasizing realistic scenarios.

Aims to develop a modular robot that consists of many reconfigurable modules and demonstrate its multifunction and reconfiguration in a desert for running, climbing, structuring, life-protecting, and flying. We have built a first generation module with a single degree of freedom and multiple connection ports on different faces.

HURT is a multi-vehicle controller that coordinates and collaboratively plans urban RSTA missions for autonomous vehicles. It implements augmented autonomy for teams of arbitrary vehicle platforms.

We present a general graph learning algorithm for spectral graph partitioning, that allows direct supervised learning of graph structures. Learning is based on gradient descent in the space of graph weights, using derivatives of eigenvectors. This algorithm effectively learns a graph capable of memorizing and retrieving multiple patterns given noisy inputs. We experimented on segmentation and recognition tasks, including bottom-up geometric shape extraction with top-down priors, and hand-written digit recognition.

This project goal is to design, develop, and implement several new algorithms and architectures for learning controllers for high-speed quadruped locomotion over rough terrain. This will be achived by incorporating a dynamically relevant lowdimensional representation of the joint trajectories for control and learning. The low-dimensional space of control parameters will be automatically learned from examples of high - dimensional joint trajectories, and these parameters will be used to compactly describe a number of primitive gaitmotions. Using a formal compositional semantics, the primitive gaits will be temporally sequenced in a hierarchical manner to generatemore complex locomotionmanuevers. Reinforcement learning techniques will be applied to optimize the switching boundaries between these primitive locomotionmodes, as well as tune the underlying low-dimensional controlparameters for speed and robustness.

We present a multiscale graph-based image segmentation algorithm. In contrast to most multiscale image processing, this algorithm works on multiple scales of the image in parallel, without iteration, to capture both coarse and fine level details. We demonstrate that large image segmentation graphs can be compressed into multiple scales capturing image structure at increasingly large neighborhood. The algorithm has O(N) time complexity, allowing to segment large images with typically N = 1000 x 1000 pixels.

We consider the problem of recovering an underwater image distorted by surface waves. Our experimental setup consists of a camera positioned above a swimming pool facing down and a book lying on the bottom of the pool. A large amount of video data of the distorted image, e.g. the cover of a book, is acquired and the problem is posed in terms of understanding the statistics of local patches in the image plane. This challenging reconstruction task can be formulated as a manifold learning problem, such that the center of the manifold is the image of the undistorted patch. To compute the center, we present a new technique to estimate global distances on the manifold.

The BIOCOMP project applies hybrid systems to modeling and simulation of metabolic and cellular control pathways. Hybrid systems combine both discrete events and continuous differential equations, unlike traditional approaches choosing exclusively between discrete or continuous dynamics. These models capture the switching behavior in phenomena such as transcription, protein-protein interactions, and cell division and growth.

The DaVinci project brings together mathematicians and engineers to study systems that can be modeled by Differential Algebraic Inequalities and Differential Complementarity Problems. The goal is to develop a set of mathematical and computational tools broadly applicable to multiple engineering disciplines, including robotics, manufacturing, chemical processes, hydraulic processes, avionics, intelligent highways, and automotive systems.

This project develops algorithms to recognize human activity from unsupervised video streams. Detection and classification address multiple levels of abstraction, including limb tracking, human identification, gesture recognition, and activity inference. The ultimate goal is to develop computational algorithms to understand human behavior in video.

Control and decision-making for independent legged robotic agents.

This research develops methodology and software for deploying multiple autonomous robots in an unstructured and unknown environment. Its framework of supervised autonomy enables both deliberate and reactive behavior for the robots during autonomous operation as they adapt to their environment and learn new tasks. It also permits a human to dynamically reprogram the robots by teleoperation. Applications span reconnaissance, surveillance, target acquisition, and removal of explosive ordnance.

In this work we employ epipolar plane image analysis to recover the positions of edge features in the scene. Once we have recovered the positions of these salient points we can use a morphing technique to synthesize new views of the scene.

Omnidirectional vision systems can provide panoramic alertness in surveillance, improve navigational capabilities, and produce panoramic images for multimedia.

This project is an effort at the GRASP Laboratory to develop a new technology in the form of a smart wheelchair. This device is equipped with a virtual interface and on-board cameras that enable the subject to navigate on the ground by interacting with the virtual system interface or use one of the built-in control algorithms.

This project dealt with the problem of recovering models of articulated figures, including humans, from single snapshots acquired with an uncalibrated camera. The resulting reconstruction algorithm can be used to recover stick figure models from newspaper photos or web site photos. It has also been used to recover models of moving figures from short video sequences.

Tele-Immersion will enable users at geographically distributed sites to collaborate in real time in a shared, environment as if they were in the same physical room. This new paradigm for human-computer interaction is the ultimate synthesis of networking and media technologies.

The main motivation for the project is to develop cooperative behavior for between unmanned aerial vehicles and or ground vehicles at the GRASP Lab. Another motivation is to develop control algorithms methodologies to allow the aircraft to form a part of a heterogeneous robot team including ground and other aerial vehicles and perform mission tasks at higher levels.