Directed Vision

Figure 1. TRICLOPS, a three camera directed vision system.

Directed Vision, often called active vision in the literature, is the purposeful direction of vision capabilities to achieve a goal. It is analogous to the difference between seeing (the sense) and looking (using the sense for a specific purpose). Directed vision intentionally moves a sensor in whatever direction is necessary to gather information about the environment so as to achieve a goal.

Normally, vision systems are stationary. Even when vision systems are on mobile platforms, there is typically no attempt to make them move in response to sensed images. We believe that moving the cameras based on processed data provides immense amounts of information.

The Real-time Intelligently ControLed Optical Positioning System (TRICLOPS) is shown in the photo above. Originally developed at NIST, TRICLOPS is on loan to the University of New Mexico for directed vision research. TRICLOPS has the following design specifications:

• 4 degrees of freedom
• 1 camera with wide angle lens for peripheral vision
• 2 cameras with telephoto lenses for foveal vision
• Moves as fast as a human eye

TRICLOPS has the following primitive capabilities:

• Large field of view with both high and low resolution
• 3 camera redundant range computation
• Real-time feature tracking
• Subpixel camera resolution

Given these primitive capabilities, we can compute models of objects in a workcell with sufficient accuracy to manipulate them. If TRICLOPS is "locked" onto a feature of interest in the world, the active vision head will follow that feature as it moves in the workspace. Our approach is to also move the sensor slightly in an appropriate direction while in the "locked" state. From this information, a feature Jacobian can be computed, allowing subpixel accuracy of both position and motion of the "locked" feature. By combining multiple feature points into simple object models, we can provide sufficient information to a robot control system to manipulate unwieldy objects.

We can apply directed vision to many DOE problems. We can build models for D & D applications. We can monitor glovebox safety quantitatively by moving a robot-mounted TRICLOPS to new viewing positions in order to avoid occlusion created by human workers. We can provide visual feedback to a robot for improved safety during task execution. Given this directed vision capability, DOE can apply this to the following applications:

• Watch the actions of the robot in the glovebox to monitor safety;
• Create an accurate world model of the important aspects of a scene, i.e., the volume where the robot is working;
• Provide feedback for the robot for improving closed loop performance of cleanup operations;
• Provide position and orientation feedback for the robot in grasping objects up the instant that the grasp is made (enhancing efficiency and safety);
• Provide an intelligent monitoring device which gathers task information by avoiding occlusion of objects of interest;
• Provide a new way to program automation actions in glovebox.

Recently, a low-cost version of TRICLOPS has been constructed. Having two cameras, it is called BICLOPS, and is shown in the figure below.

Figure 2. Biclops, a two camera directed vision system.

For additional information, please contact
Ron Lumia, Professor
Department of Mechanical Engineering
University of New Mexico
Albuquerque, NM 87131
505-272-7155
lumia@unm.edu