Category Archives: Research

Robotic Assembly by Intelligent Precision Jigging Robots

Assembly robots have found practical use on assembly lines for decades, predictably performing the same routine motions in a streamlined process that has made advanced technology cheap and widely available. Outside of the assembly line, robotic assembly has never advanced past the prototype and toy problem phase.

Robots that can replace humans on construction projects have long been desired, but progress toward that goal has been limited. One notable application is construction in hazardous environments that humans cannot easily access. Making this practical requires the intersection of numerous subfields, including sensor fusion, distributed algorithms, motion planning, error detection and correction, and sequence planning. Construction calls for the ability to handle a wide range of issues, including cutting and shaping parts as needed, addressing assembly mistakes, distributing tasks to specialized agents, and handling unexpected events. Accuracy requirements call for sensors and optimal estimation algorithms, especially when an assembly does not consist of interlocking, self-correcting linkages. Complex and expensive robots that can handle all tasks are particularly risky, as the loss of one can be devastating. Reliance on pre-made, precisely machined parts can also lead to failure if just one unique part breaks. Often, the attachment of a subassembly requires several agents coordinating, for example, to hold up and maneuver large parts.

Intelligent Precision Jigging Robots

To solve this, I introduced Intelligent Precision Jigging Robots (IPJRs) as a solution. Intelligent Precision Jigging Robots are robots that can precisely hold a pose between disconnected parts (in welding, this is known as jigging), enabling an external agent (such as a robotic arm) to permanently bond the parts. IPJRs adjust the distances between the parts using highly precise actuators and precise sensors for distance measurement. IPJRs free the external agents from having a high precision requirement, reducing the complexity of the external agent. A set of IPJRs and external agents performing assembly are individually simpler than an all-purpose assembly robot, thus they can be made more inexpensively, enabling robustness.

IPJRs attach to nodes and position a strut of variable length between them.
IPJRs attach to nodes and position a strut of variable length between them. This repeats cell by cell until the structure is finished.
NASA - James Webb Space Telescope (left), 20 meter far-infrared concept.
NASA – James Webb Space Telescope (left), 20 meter far-infrared concept (right).

My thesis research focused on methods to enable the cheap construction of space telescopes using only raw/stock materials by distributed robots. Space telescopes, through their stringent precision requirements, represent a superset of truss requirements, so that anything shown to work on telescope assembly should work on other kinds of structures.

Prototype 1: 2D Truss Assembly One Cell At a Time

Prototype 1 was a proof of concept prototype developed shortly after the commencement of my research. I began designing the prototype while a Visiting Fellow at Harvard with with the Self-Organizing Systems Research Group in the summer of 2012, and continued through the fall.

Prototype 1 triangle with glue gun and wooden dowels.
Prototype 1 triangle with glue gun and wooden dowels.
Large irregular ring made by Prototype 1 in December 2012.
Large irregular ring made by Prototype 1 in December 2012.

The three IPJRs were arranged in a triangle controlled by a single Arduino, and each edge’s length was adjusted by a Firgelli L16 actuator. The actuators provided length measurements via an onboard potentiometer, which was used to guide a rudimentary controller. When actuated, the triangle positioned three wooden nodes to a precision of 0.5mm, and an external welding agent (me, with a glue gun) bonded a wooden strut between the nodes. I ran several trials of a square, and one of a large irregular ring, to show that the mean error in node-to-node distance was about 0.5mm.

Prototype 2: 2D High Precision Assembly at Langley Research Center

Prototype 2 was built to demonstrate telescope truss assembly using materials and hardware similar to what would be used in a flight mission. The second IPJR prototype was designed to align the tops of the node posts (where a mirror would be attached) to within 5 microns of a set length in the range of 0.987 to 1.013 meters. It used Ultra Motion D-Series linear actuators with 7.9 micron steps and a 2 inch stroke length. To test repeatability and to calibrate, each edge had a Keyence IL-030 laser distance sensor to sense the length of the IPJR edge by measuring the extension of the Ultra Motion actuator. The IL-030 has a repeatability of 1 micron, but a small operational range of 20-45 mm, which limited the range of the IPJR.

The IPJR was a triangle consisting of three identical edges. Each edge is a mechanical linkage between two components, the main body and the node gripper module. The main body of each edge was a composite tube, a material chosen for its favorable thermal expansion properties over most metals. An Ultra Motion actuator extended the node gripping module away from the main body. Two rails with bearings were used to prevent bending moments on the Ultra Motion actuators, which can cause them to fail under lateral loads of just 13 Newtons.

The node gripping module consisted of a funnel used for guiding a node post while the IPJR is being lowered onto the canister, a target plate for capturing node posts precisely, and a stepper motor to push the node post into the target plate. The IL-030, attached to the main body, measured the edge’s extension by measuring its distance to the target plate. Attached to the tube were two grippers for lifting and holding the struts. To eliminate free play within the entire IPJR, springs were attached between the lifting plate posts, imposing a compressive force on each edge.

The LSMS prepares to weld a strut to the center node.
The LSMS prepares to weld a strut to the center node.

The Lightweight Surface Manipulation System (LSMS) is a long-reach manipulator designed by my colleagues at Langley Research Center to operate on planetary surfaces, intended to manipulate large objects in preparation for a manned landing. It is 4.25 meters tall and has a reach of 8.5 meters. An off-the-shelf arc welding gun was integrated into an end effector attached to the LSMS wrist. A lifting plate end effector gave the LSMS the ability to lift the IPJR by grasping three attachment points on the IPJR. The limited maneuverability of the LSMS required a turntable to provide the final degree of freedom necessary for controlled placement of the IPJR and the welding gun. The canister presented new struts and nodes for the IPJR to capture while the LSMS positioned the IPJR over the canister.

The second prototype constructed a 2D truss made of 7 titanium nodes and 12 titanium struts, arranged in a hexagonal lattice of six cells, representing a simplified optical bench for a space telescope. The cells were nominally equilateral triangles with a node-to-node distance of 1.002 meters. The assembly experiment took place in September 2013.

Prototype 2 rests on completed structure.
Prototype 2 rests on completed structure.

The demonstration of truss assembly and welding by robotic agents was considered a complete success. However, the precision of the final structure did not meet the precision of the assembly hardware; the node positions ranged in error from 3.2 to 5.0 mm. I decided that the third and final prototype needed to have an active system for detecting and correcting errors during assembly.

Prototype 3: 3D Truss Assembly with Error Detection and Correction
A Prototype 3 IPJR, shown attached to a strut between two nodes.
A Prototype 3 IPJR, shown attached to a strut between two nodes.

Prototype 3 was designed to perform repeatable 3D assembly experiments using disconnected IPJRs (that is, not arranged in prebuilt triangles as with the previous two prototypes) and an error detection and correction algorithm. Struts were made of 30 inch telescoping aluminum rods, 1/4 inch and 7/32 inch diameter, lockable by shaft collars. At each end were neodymium magnets that connected to 1.5 inch steel node balls.

A node ball with numerous struts attached by neodymium magnets.
A node ball with numerous struts attached by neodymium magnets.

I made 5 IPJRs, but only needed 3 at a time; the others were backups. Each IPJR was an autonomous robot, consisting of a Raspberry Pi Model B for high level algorithmic control and communications, an Arduino for actuation and sensing, a custom motor driver board, and an Edimax WiFi dongle. The principal actuator was a Firgelli L16 actuator, which was also used on Prototype 1. To attach to both tubes of a strut, each IPJR had two shaft collars, enabling the actuator to extend and contract the strut. The IPJRs communicated to one another, and a central control PC, via HTTP packets over the wireless network. The control PC both planned a sequence and used Maximum Likelihood Estimation or an Extended Kalman Filter to estimate error and correct future assembly steps. Also, like Prototype 1, this experiment did not use a robotic external manipulator, so I filled that role.

I ran two sets of experiments, each building a (different) variant of a 3D space telescope optical bench with surface curvature. The first set of experiments used the Extended Kalman Filter for state estimation and correction, while the second set of experiments used the Maximum Likelihood Estimator.

Completed truss using MLE.
Completed truss using MLE.

The next stage of my assembly research is in progress at Langley Research Center – I will reveal more when I can!

Acknowledgements

My thesis research was funded by NASA Space Technology Research Fellowship #NNX11AM83H.

  • Nikolaus Correll: for advising me.
  • Eric Frew, Sriram Sankaranarayanan, Tom Yeh, and Dan Scheeres: for serving on my committee.
  • Dustin Reishus: for mathematical contributions.
  • Andy McEvoy: for assembly stability research and numerous discussions.
  • John Dorsey and Bill Doggett: for help refining the IPJR concept and bringing me onto their team.
  • Dave Mercer, Bruce King, and Rob Hafley: for help designing and executing the Prototype 2 experiment at Langley Research Center.
  • Radhika Nagpal: for discussions that led to the design of Prototype 1.

Intelligent Exploration of Reachability Sets

Space mission trajectories require extensive and careful planning due to limited fuel budgets and the complexity of gravitational dynamics. For many applications, planners decompose the problem into smaller segments and solve each one using a simplified version of the dynamics. This works well for maneuvers such as gravity assists and in systems with dominant single bodies that can be approximated by point masses, but it fails in systems with highly irregular bodies. To automatically plan trajectories in these systems, one must model the full nonlinear dynamics, and do so ‘on the fly,’ in order to handle unpredictable effects due to orbit uncertainty and the complex nonlinear dynamics of gravitation. However, the full set of possible trajectories is infinite, and is often chaotic, meaning that the evolution of any two trajectories, no matter how close, will diverge exponentially in finite time. Exploration must then be done by looking ahead intelligently.

NASA - Pluto and Charon as seen by New Horizons.  The Pluto-Charon system is a multi-body system with nonlinear dynamics.
NASA – Pluto and Charon as seen by New Horizons. The Pluto-Charon system is a multi-body system with nonlinear dynamics.

My colleagues and I developed the Reachable Set Explorer, an AI technique that randomly explores the space of possible maneuvers for a spacecraft for a finite time horizon. The set of simulated trajectories is subdivided into ‘interesting’ regions (where ‘interesting’ can be any metric of the user’s choice), which contribute a higher weight for the next round of random choices. The result is an approximate reachability set that is more precise and detailed around the more interesting regions. The cost of simulating each trajectory is the driving factor in time, so the explorer must find a good approximation using a fixed number of trajectories.

In our published research, ‘interesting’ trajectories are those that straddle the boundary between impacting a body and skimming the surface. Given the chaotic nature of gravitational systems in general, finding orbits that make multiple low passes over the various bodies while minimizing fuel expenditure can potentially produce highly useful data.

Conceptually, the RSE algorithm can be thought of as a mixture between mesh refinement algorithms and shooting methods for nonlinear control systems. Given a fixed number of trajectories to create an approximate mesh, the algorithm seeds the space of possible trajectories randomly with a small fraction of the desired total. A mesh is built using a Delaunay triangulation of the seeds. Using weighted random sampling, the algorithm picks simplices whose vertices have different fates with a higher likelihood, then inserts new trajectories randomly into the simplices, and rebuilds the mesh. This process continues until the total number of trajectories has been calculated. Our research explored the parameter space – how to seed, how to choose places to add new meshes (e.g. random sampling and placement is more likely to find hidden regions), and so on.

Exploring the reachability set randomly shows low detail on the boundaries and excessive exploration of impact zones (red and green) and escape zones (yellow).  Blue indicates trajectory survival over a finite time horizon.
Exploring the reachability set randomly shows low detail on the boundaries and excessive exploration of impact zones (red and green) and escape zones (yellow). Blue indicates trajectory survival over a finite time horizon.
A mesh made by the RSE algorithm uses the same number of vertices, but better reveals the details of the boundary between impact and continuation.
A mesh made by the RSE algorithm uses the same number of vertices, but better reveals the details of the boundary between impact and continuation.
Zooming in on the spiral in the lower right of the full set reveals additional detail.
Zooming in on the spiral in the lower right of the full set reveals additional detail.

One of the interesting things to become visible through our explorations is the fractal nature of orbital mechanics. The best meshes concentrate points in areas where the trajectories take highly variable paths before terminating. In the plots shown here, these areas look like condensed colored zones – slight changes result in escape, impacting the green body, or the red body. Zooming in reveals more of the same, and I conjecture that one will se similar formations at arbitrary zoom levels – the key feature of a fractal. In chaotic systems, an infinite number of periodic trajectories exist, including orbits that visit one body X times, followed by another Y times, and so on.

My involvement in this research ended after the foundational stage. It has since been picked up by David Surovik, a fellow NSTRF fellow, under Dan Scheeres in the Celestial and Spaceflight Mechanics Laboratory.

Our published papers are below:

Intelligent Computation of Reachability Sets for Space Missions, IAAI, 2012

Download (PDF, 588KB)

Efficiently Locating Impact and Escape Scenarios in Spacecraft Reachability Sets, AIAA Astrodynamics Specialists, 2012

Download (PDF, 4.75MB)

Efficiently Evaluating Reachable Sets in the Circular Restricted 3-Body Problem, IEEE Transactions on Aerospace and Electronic Systems, 2014

Download (PDF, 6.21MB)

IPJR Papers

I have published six papers on the IPJRs. An article for the International Journal of Robotics Research is currently under review. See CV page for full bibliography.

Precise Assembly of 3D Truss Structures Using EKF-Based Error Prediction and Correction, ISER, 2014

Download (PDF, 1.29MB)

Truss Assembly and Welding by Intelligent Precision Jigging Robots, TePRA, 2014

Download (PDF, 4.33MB)

Assembly Path Planning for Stable Robotic Construction, TePRA, 2014

Download (PDF, 861KB)

Precise Truss Assembly Using Commodity Parts and Low Precision Welding, Journal of Intelligent Service Robotics, 2014 (extends TePRA 2013 paper).

Download (PDF, 5.2MB)

Precise Truss Assembly Using Commodity Parts and Low Precision Welding, TePRA, 2013.

Download (PDF, 2.47MB)

An Efficient and Versatile Means for Assembling and Manufacturing Systems in Space, SPACE, 2012.

Download (PDF, 1.82MB)

Curriculum Vitae

Thesis

Precise Assembly of Truss Structures by Distributed Robots, PhD Dissertation, University of Colorado at Boulder, 2014.

Journal Articles

Erik Komendera and Nikolaus Correll. Precise Assembly of 3D Truss Structures Using MLE-Based Error Prediction and Correction. International Journal of Robotics Research, Under Review, 2015.

Erik Komendera, Joshua Garland, Elizabeth Bradley, and Daniel J. Scheeres. Efficiently Evaluating Reachable Sets in the Circular Restricted 3-Body Problem. IEEE Transactions on Aerospace and Electronic Systems 51:454-467, 2015.

Erik Komendera, Dustin Reishus, John T. Dorsey, William R. Doggett, and Nikolaus Correll. Precise Truss Assembly Using Commodity Parts and Low Precision Welding. Journal of Intelligent Service Robotics, 2014.

Conference Papers

Erik Komendera, William R. Doggett, and John T. Dorsey. Control System Design Implementation and Preliminary Demonstration for a Tendon Actuated Lightweight In-Space MANipulator (TALISMAN), to appear in Proceedings of the AIAA SPACE Conference, 2015.

Erik Komendera and Nikolaus Correll. Precise Assembly of 3D Truss Structures Using EKF-Based Error Prediction and Correction. In Proceedings of the 2014 International Symposium on Experimental Robotics, 2014.

Erik Komendera, John T. Dorsey, William R. Doggett, and Nikolaus Correll. Truss Assembly and
Welding by Intelligent Precision Jigging Robots. In Proceedings of the Sixth Annual IEEE International Conference on Technologies for Practical Robot Applications, 2014.

Michael A. McEvoy, Erik Komendera, and Nikolaus Correll. Assembly Path Planning for Stable Robotic Construction. In Proceedings of the Sixth Annual IEEE International Conference on Technologies for Practical Robot Applications, 2014.

Erik Komendera, Dustin Reishus, John T. Dorsey, William R. Doggett, Nikolaus Correll. Precise Truss Assembly using Commodity Parts and Low Precision Welding. In Proceedings of the Fifth Annual IEEE International Conference on Technologies for Practical Robot Applications, 2013.

John T. Dorsey, William R. Doggett, Erik Komendera, Nikolaus Correll, Robert Hafley, and Bruce D. King. An Efficient and Versatile Means for Assembling and Manufacturing Systems in Space. In Proceedings of the AIAA SPACE Conference, 2012.

Erik Komendera, Elizabeth Bradley, and Daniel J. Scheeres. Efficiently Locating Impact and Escape Scenarios in Spacecraft Reachability Sets. In Proceedings of the AIAA Astrodynamics Specialists Conference, 2012.

Erik Komendera, Daniel J. Scheeres, and Elizabeth Bradley. Intelligent Computation of Reachability Sets for Space Missions. In Proceedings of the Innovative Applications of Artificial Intelligence Conference, 2012.

Posters

Erik Komendera, William R. Doggett, and John T. Dorsey. Tendon Actuated Lightweight In-Space MANipulator (TALISMAN) To appear at The Next Generation of Space Robotic Servicing Technologies Workshop, IEEE International Conference on Robotics and Automation, 2015.

My CV:

Download (PDF, 74KB)

My PhD dissertation, defended November 7, 2014:

Download (PDF, 25.43MB)

Collaborators

This page lists some of the people with whom I have done research.

Correll Lab

My advisor was Nikolaus Correll – he is an Assistant Professor at the University of Colorado at Boulder. Their research is mostly in swarm robots and smart materials. He has been in the press recently:

I co-authored a paper with Andy McEvoy on assembly stability. He has since gone on to smart materials, and has recently had an article featured in Science.

I co-authored a few papers with former post-doc Dustin Reishus, who was instrumental in helping to mathematically formalize what *was* Intelligent Scaffolding, and is now Intelligent Precision Jigging Robots. He went to Google in 2013.

I have no common papers with Michael Otte, but we have had countless discussions about math and algorithms. Multi-robot path planning is his specialty. The following video demonstrates a tiny robot path planning with the RRT-X algorithm while dodging Pac-Men and saw blades.

NASA

John Dorsey was my NSTRF mentor, and now I work in his robotics group at Langley Research Center. Bill Doggett has also been a key mentor for my NASA work. The following video shows Dorsey, Tom Jones, and Judith Watson assembling the space telescope truss that served as the motivation for my dissertation.

Vytas SunSpiral, a senior roboticist at Ames Research Center, works on tensegrity robots. I first met him in 2012, and have been looking for a way to collaborate with him ever since. His latest work is the SuperBall Bot, a proposed tensegrity robot that can land as-is on Mars, absorb the impact with its topology, and then start rolling around on the surface. Watch the video below to see the SuperBall Bot.

Reachability Sets

Liz Bradley, CS Professor at CU-Boulder, does research in AI and nonlinear dynamics. Our Reachability Set research grew out of a class project in her Chaotic Dynamics class.

Dan Scheeres is an Aerospace Engineering Sciences professor at CU-Boulder. He was a professor at Michigan when I was an undergrad, and I took one of his courses, but we did not start working together until Colorado.

Joshua Garland briefly took over the Reachability Set research, which required him to make sense of all my old code and redo it to run more nicely.