Videos
Loading the player ...
  • Offer Profile
  • In the Telerobotics Lab, we're interested in robotic systems that manipulate remote environments. A primary focus of our work is medical telerobotics. Particularly, we are investigating wireless magnetically controlled microrobots to navigate inside the human body. Rather than acting as autonomous systems, our microrobots act more like end-effectors of novel teleoperation system unlike anything that has ever existed. For decades we have been imagining the day when we could perform minimally invasive medical procedures from inside the body. We're actually getting there!
Product Portfolio

  • Untethered Magnetic Biomedical Microrobots

  • One approach to the wireless control of microrobots is through externally applied magnetic fields. These untethered devices could navigate bodily fluids for minimally invasive surgical and diagnostic procedures, or could be used as the end-effectors of micromanipulation systems.
      • 1. Helical Magnetic Microswimmers
        There is a class of untethered microrobots that use helical propellers (similar to a corkscrew) to swim using a method inspired by the propulsion of bacteria. If a piece of magnetic material is attached to the helical propeller, then magnetic fields can be used to rotate the microrobot for propulsion, as well as to steer the microrobot.
        2. Control Using Field Gradients for Intraocular Microrobots
        Magnetics is a complicated discipline, and most magnetics problems are solved using numerical methods. However, these are often not appropriate for use in real-time magnetic control. We have developed models such that we can now calculate the torque and force generated by an arbitrary magnetic field on an axially symmetric soft-magnetic body. We consider the magnetization of the body as a function of the applied field, using a continuous model that unifies two disparate magnetic models. The continuous torque and force follow. The model is verified experimentally, and captures the often-neglected region between weak and saturating fields, where interesting behavior is observed. We can calculate the field direction to maximize torque for a given field magnitude. We also find an absolute maximum torque, for a given body geometry and material, which can be generated with relatively weak applied fields. We have extended the model to consider the real-time control of assembled-MEMS devices that may have significantly more complex geometries. We validated the model through FEM and experiments. The model captures the characteristics of complex 3-D structures and allows us, for the first time, to consider full 6-DOF control of untethered devices.

  • Robotic Assistance for Cochlear Implants

      • Magnetic Steering of Cochlear Implants

      • The goal of this project is twofold: to improve the safety of cochlear implant procedures, and to improve the functionality of the implant. A cochlear implant is an array of electrodes surgically inserted into the cochlea. Its purpose is to recreate hearing in those who are deaf. During the implant surgery, there are a number of things that can go wrong. Even with successful implants, there is a lot of room for improvement, including inserting the implant deeper into the cochlear, which will lead to hearing lower-frequency tones, as well as curling the implant tighter around the nerve, which will result in better frequency discrimination as well as improved power efficiency.

        Our goal is to use magnetic fields to steer the tip of the cochlear implant as it is inserted into the cochlea. The image of our experimental setup shown here shows one robotic stage that inserts a to-scale implant prototype into an Advanced Bionics to-scale cochlear implant phantom, as an ATI Nano17 force sensor measures the insertion force. A second robotic stage moves a large permanent magnet toward and away from the cochlea, as a motor rotates the magnet in sync with the implant insertion.

      • Scala-Tympani Phantoms for In Vitro Insertion Experiments

      • For cochlear-implant insertion experiments, such as in the development of the proposed magnetically steered implants, it is desirable to have a phantom scala tympani for laboratory experimentation. We have developed a model based on published anatomical data that can be used to create a scala-tympani phantom at any desired scale. The results will be published in the ASME Journal of Medical Devices. Here is the Matlab file used to generate the model, as well as the SolidWorks model that can be used to create a 3:1 scale phantom. Our fabricated ST phantom is shown in the image.

  • Haptic Interaction with Admittance-Type Devices

      • Haptic devices are typically robots of the impedance type, meaning that they are designed to have very small amounts of inertia and friction, such that the human user experiences only the forces output by the virtual environment. Admittance-type robots are heavily geared with significant amounts of inertia and friction, which makes them well suited for slow, precise movements, but they are rarely used as haptic devices. When they are used as haptic devices, the human interacts with a force sensor on the device, and this sensed force is commanded into controlled movement, leading to a significantly different experience than when using an impedance-type device. Admittance-type robots have proven to be effective as robot-assistance devices, such as the Johns Hopkins University Steady-Hand microsurgical robot, and more recently the University of Utah Active Handrest. Admittance-type devices also show promise as human input devices for precise master-slave telemanipulation systems.

        We are exploring how humans interact with admittance-type device during precise manipulation tasks. Specifically, we are considering how the allowable speed of the device, as well as the control-system implemented, affect the user's ability to precisely control the movement of the device.

  • Avian-Inspired Graspers

      • Songbirds perch using a tendon that causes the foot to close when the leg collapses. We are developing graspers inspired by the feet and legs of songbirds that will enable flying robotic rotorcraft such as helicopters and quadrotors to perch like a bird.

  • Rehabilitation in the Treadport

  • The Treadport is an immersive virtual environment at the University of Utah. We are exploring how the Treadport might be used in the rehabilitation of patients recovering from spinal-cord injuries. 

  • How to Build an Inexpensive 5-DOF Haptic Device using Two Novint Falcons

      • On this page we demonstrate how two Novint Falcons, inexpensive commercially available haptic devices, can be modified to create a reconfigurable five-degree-of-freedom (5-DOF) haptic device for less than $100 (in addition to the price of the two Falcons, which at last check were selling for $250). The device is intended as an educational tool to allow a broader range of students to experience force and torque feedback, rather than the 3-DOF force feedback typical of inexpensive devices. Commercially available 5- and 6-DOF devices exist and feel great, but they can easily run $50,000 or more. Our 5-DOF device doesn't feel as nice, but at 1% of the price, it can be a great educational and research tool. Plus, all of the modifications described here are nondestructive, so you can always easily revert your Falcons back to their original condition.

        The pictures on the left show the different configurations of the 5-DOF device. Each configuration uses the same stylus, so changing configurations is quick and easy. Since the Novint Falcons have limited range of motion, the best stylus configuration must be chosen for a given application. On this page, we explain how to build the device for yourself, we explain how to implement a 5-DOF force/torque control system that makes the two Falcons act in concert as a single haptic device, and we explain how to implement gravity compensation to make the now-even-heavier Falcons feel light. The gravity-compensation algorithm can also be applied to standard Falcons without our 5-DOF modifications.

        Through the course of this project, we developed a gravity-compensation routine for the Falcon. Even if you don't want to build the modified 5-DOF device, the addition of gravity compensation makes the heavy Falcon feel much nicer. The publication and software below include code for both the 5-DOF and original 3-DOF device.

  • Self-Assembling Swallowable Modular Robots

  • This is work conducted in Brad Nelson's lab at ETH Zurich.
      • Capsule endoscopes such as the M2A by Given Imaging are gaining popularity is clinical use. Capsule endoscopes are swallowable cameras that move passively through the GI tract to diagnose problems such as bleeding and cancer. A great deal of robotics researchers are working towards adding functionality to capsule endoscopes, such as legs to provide controlled locomotion, or tools to diagnose pathologies. However, adding more functionality is difficult without making the capsule larger, and consequently more difficult to swallow.

        As part of the ARES project, we are working towards a swallowable modular robotic system. The goal is to make a system composed of small simple components that are easy to swallow, and that assemble inside the stomach or small intestine to form a larger structure with the desired functionality. Our research has largely focused on magnetic self-assembly.

  • Stable Haptic Interaction

  • This research was conducted in Allison Okamura's lab at Johns Hopkins University.

      • Effects of Position Quantization and Sampling Rate on Virtual-Wall Passivity

      • A virtual wall is the most common building block used in haptic virtual environments. It is typically implemented as a spring or spring-damper surface; when the haptic device penetrates past the virtual wall, a force is commanded that is proportional to the depth of penetration. To make a virtual environment feel like real materials (steel, wood, etc.) we would like to make virtual walls feel very stiff. However, making virtual walls too stiff can lead to instabilities where the haptic device vibrates against the virtual wall, destroying any illusion of reality.
        A passive system is a system that is incapable of generating energy (that is, it can only dissipate and store energy). All inanimate objects in the natural world are passive, and humans know how to interact with them in a natural way. The thought is, if we make robotic systems that are passive, then humans will know how to naturally interact with them as well.
        Previous work has considered the interplay of sampling rate (of the computer and A/D running the haptic virtual environment) and viscous friction in the joints of the haptic device. In this work, we take the next step, and additionally consider both the effects of position quantization (due to measuring robot position with an optical encoder, which is standard practice) and Coulomb friction (making the overall friction model much more accurate for modeling motors, etc.). We consider the haptic device, implementing a basic spring virtual wall, using a model that relates the force applied by the user fh to the position of the haptic device x.
        The result of our analysis is a simple condition to guarantee passivity of the virtual wall system shown:
        K < min(2b/T , 2fc/Delta)
        where K is the virtual wall stiffness in {N/m}, b is the viscous friction in {Ns/m}, fc is the Coulomb friction in {N/m}, and Delta is the resolution of the encoder in {m}. The most important result of this research is that the term based on Coulomb friction and encoder resolution is typically the dominant (i.e., smaller) term, making the consideration of these effects very important for accurate passivity analysis.

      • Stable Forbidden-Region Virtual Fixtures

      • A forbidden-region virtual fixture is constraint, implemented in software, that helps keep a robotic manipulator out of forbidden regions. Forbidden-region virtual fixtures have the potential to increase the safety of telemanipulated tasks. The problem is, stiff virtual fixtures (that is, those that are actually effective in keeping the slave robot out of forbidden-regions) can demonstrate unstable vibrations. Our research addressed this stability problem, and presented a new method to predict (and consequently, prevent) this instability.

  • Haptic Virtual Fixtures for Task Assistance in Telemanipulation

  • This research was conducted in Allison Okamura's lab at Johns Hopkins University.
      • Teleoperation, which is the control of remote robots, can be used to accomplish a great number of tasks that are too remote, highly scaled, or hazardous for direct human manipulation. It is particularly advantageous in unstructured environments, where completely autonomous robotic systems cannot be used due to the limitations of artificial intelligence, sensor data interpretation, and environment modeling. "Bilateral telemanipulation" typically refers to a teleoperation system where a human operator manipulates a "master" robotic device, and a "slave" device emulates the behavior of the master, with some form of haptic (force and/or tactile) feedback to the operator. Although haptic feedback can improve performance of telemanipulated tasks, traditional teleoperation systems are not able to provide any intelligent assistance to the human operator. Thus, we are developing novel human-machine collaborative control laws, called "virtual fixtures," which bridge the gap between strictly autonomous and strictly teleoperated systems. The term "virtual fixture" refers to a general class of guidance modes, implemented in software, that help a human-machine collaborative system perform a task by limiting movement into restricted regions and/or influencing movement along desired paths. The potential benefit of virtual fixtures is safer and faster operation. Virtual fixtures attempt to capitalize on the accuracy of robotic systems, while maintaining a degree of operator control.

        I worked with master/slave teleoperators. In telemanipulation, a user operates a robot that is at some distant location (it could be thousands of miles, or in the same room). In master/slave telemanipulation, rather that controlling the distant (slave) robot with a joystick or a keyboard, the user moves an identical (or at least similar) master robot, and the slave then follows its movement.

  • Pulse-Frequency-Modulation Control of Neuroprostheses

  • This is research done with Sanford Meek at the University of Utah.
      • The goal of the project is to interface a prosthetic arm directly to the peripheral nervous system of an amputee. The amputee will then control the arm just like he/she would control a real arm. The Bioengineering department at the University of Utah has developed electrodes that can be implanted directly into a truncated nerve. These sensors can be used to measure efferent signals coming from the brain down to the arm, and to input afferent signals back into the nervous system. The popular electromechanical prosthesis Utah Arm 2 from Motion Control has been modified to be controlled with neural signals, as opposed to myoelectric signals that are currently uses (EMG signals measured from the surface of muscles). It has also be given extra sensors and actuated degrees of freedom.

        The nervous system uses a method of transmitting information known as pulse frequency modulation (PFM). In PFM, the information in a signal is carried by the instantaneous frequency of a pulse train, and the pulses themselves are essentially identical. This figure shows an example of the input/output behavior of a pulse frequency modulator. The nervous system uses this type of signal because of the way voltages are generated at the cellular level. However, PFM signals are rarely used in engineering applications, and systems that contain PFM are consequently not well understood.
        We considered methods of modulating a PFM signal (that is, creating a pulse train from a continuous signal) and methods of demodulating a PFM signal (that is, trying to reconstruct the original continuous signal from a pulse train). We developed methods to analyze the stability of closed-loop systems that use PFM signals, and methods to predict the amplitude and frequency of undesirable limit cycles. We also developed a method to generate emulate a pulse frequency modulator digitally.