Brown University
Loading the player ...
  • Offer Profile
  • The Brown Robotics Group strives to realize robots and autonomous systems that are effective collaborators for pursuing human endeavors. Towards this end, we explore problems in human-robot interaction, robot learning, robot perception, autonomous control, dexterous manipulation, and game development. Our solutions to these problems have applications including human-robot teams, humanoid robotics, robot soccer, and neural prostheses.
Product Portfolio
  • Projects

    • Physical Simulation for Probabilistic Motion Tracking

    • We explore the use of full-body 3D physical simulation for human kinematic tracking from monocular and multi-view video sequences within the Bayesian filtering framework. Towards greater physical plausibility, we consider a human's motion to be generated by a ``feedback control loop'', where Newtonian physics approximates the rigid-body motion dynamics of the human and the environment through the application and integration of forces. The result is more faithful modeling of human-environment interactions, such as ground contacts, resulting from collisions and the human's motor control.
    • Multi-robot Markov Random Fields

    • We explore Markov random fields (MRFs) as a probabilistic mathematical model for unifying approaches to multi-robot coordination or, more specifically, distributed action selection. We describe how existing methods for multi-robot coordination fit within an MRF-based model and how they conceptually unify. Further, we offer belief propagation on a multi-robot MRF as an alternative approach to distributed robot action selection.
    • Learning Robot Soccer from Demonstration

    • We seek to enable users to teach personal robots arbitrary tasks, allowing robots to better serve users' wants and needs without explicit programming. Robot learning from demonstration is an approach well-suited to this paradigm, as a robot learns new tasks in new environments from observations of the task itself. Many current robot learning algorithms require the existence of basic behaviors that can be combined to perform the desired task. However, robots that exist in the world for long timeframes may exhaust this basis set. In particular, a robot may be asked to perform an unknown task for which its built in behaviors may not be appropriate.
    • Sparse Control for Manipulation

    • Human control of high degree-of-freedom robotic systems is often difficult due to the overwhelming number of variables that need to be specified. Instead, we propose the use of sparse subspaces embedded within the pose space of a robotic system. Driven by human motion, we addressed this sparse control problem by uncovering 2D subspaces that allow cursor control, or eventually decoding of neural activity, to drive a robotic hand. Considering the problems in previous work related to noise in pose graph construction and motion capture, we introduced a method for denoising neighborhood graphs for embedding hand motion into 2D spaces. Such spaces allow for control of high-DOF systems using 2D interfaces such as cursor control via mouse or decoding of neural activity. We present results demonstrating our approach to interactive sparse control for successful power grasping and precision grasping using a 13 DOF robot hand.
    • Manipulation Manifolds

    • We explore the use of manifold learning techniques to uncover structure in sensorimotor time series from teleoperated humanoid manipulation tasks. Data from Robonaut, NASA's humanoid robot, was recorded while it was being teleoperated through four tool manipulation tasks. We show that one algorithm, Spatio-Temporal Isomap, is capable of uncovering behavioral structures that can be difficult to find with other dimension reduction techniques (Principal Component Analysis, Multidimensional Scaling, and Isomap).
    • Human Pose and Action Recognition for Interactive Robots

    • There is currently a division between real-world human performance and the decision making of socially interactive robots. Specifically, the decision making of robots needs to have information about the decision making of its human collaborators. This circumstance is partially due to the difficulty in estimating human cues, such as pose and gesture, from robot sensing. Towards crossing this division, we present a method for kinematic pose estimation and action recognition from monocular robot vision through the use of dynamical human motion vocabularies.
    • Sketch-based Mesh Animation

    • We present a methodology for articulating and posing meshes, in particular facial meshes, through a 2D sketching interface. Our method establishes an interface between 3D meshes and 2D sketching with the inference of reference and target curves. Reference curves allow for user selection of features on a mesh and their manipulation to match a target curve. Our articulation system uses these curves to specify the deformations of a character rig, forming a coordinate space of mesh poses. Given such a coordinate space, our posing system uses reference and target curves to find the optimal pose of the mesh with respect to the sketch input. We present results demonstrating the efficacy of our method for mesh articulation, mesh posing with articulations generated in both Maya and our sketch-based system, and mesh animation using human features from video. Through our method, we aim to both provide novice-accessible articulation and posing mesh interfaces and rapid prototyping of complex deformations for more experienced users.
    • World Space Motion Control for Humanoids

    • Dynamo (DYNAmic MOtion capture) is an approach to controlling animated characters in a dynamic virtual world. Leveraging existing methods, characters are simultaneously physically simulated and driven to perform kinematic motion (from mocap or other sources). Continuous simulation allows characters to interact more realistically than methods that alternate between ragdoll simulation and pure motion capture.

      The novel contributions of Dynamo are world-space torques for increased stability and a weak root spring for plausible balance. Promoting joint target angles from the traditional parent-bone reference frame to the world-space reference frame allows a character to set and maintain poses robust to dynamic interactions. It also produces physically plausible transitions between motions without explicit blending...

    • The SmURV robotics platform

    • The SmURV platform (Small Universal Robotics Vehicle) is a comparatively cheap and easy to assemble robotics platform for educational, research and hobby purposes. Essentially, the platform consists of an iRobot Create and a small computer mounted on top of it. Due to this very simple design, using components available at any larger electronics store, we are able to build an autonomous robot in a very short amount of time and can focus on the real interesting part: Namely, Making the robot do something exciting! In the following paragraphs we show what parts are needed to build a SmURV, how much they approximately cost and how they have to be assembled. Further, we offer one particular (out of many possible) software solutions to control a SmURV and to write software for it.
    • Robot Learning and Gaming

    • Towards accessible teaching of robots from demonstration, we have developed a mixed-reality distributed multi-player robotic gaming environment. Our goal is to provide robot learning researchers with an means to collect large corpora of data representative of human decision making. Robot control by an human operator (or teleoperation) is cast in a video game style interface to leverage the ubiquity and popularity of games while minimizing tedium in robot training.
    • Discovering Categories in Robot Sensor Data

    • We address the symbol grounding problem for robot perception through a data-driven approach to deriving categories from robot sensor data. Unlike model-based approaches, where human intuitive correspondences are sought between sensor readings and features of an environment (corners, doors, etc.), our method learns intrinsic categories (or natural kinds) from the raw data itself. We approximate a manifold underlying sensor data using Isomap nonlinear dimension reduction and apply Bayesian clustering (Gaussian mixture models) with model identification techniques to discover categories (or kinds). We demonstrate our method through the learning of sensory kinds from trials in various indoor and outdoor environments with different sensor modalities. Learned kinds are then used to classify new sensor data (out-of-sample readings). We present results indicating greater consistency in classifying sensor data employing mixture models in non-linear low-dimensional embeddings.