The system components:

• 19 DOF kinematics model of (right) hand/arm .
• Trajectory planner

Reach planner (Inverse kinematics solver)

Grasp planner (affordance selection)

• 3D graphics engine
• Action Recognition unit

MOUSE

 By dragging (click and move the moise pointer) the workspace you can change your 3d viewpoint at anytime.

First one from the right is used for zoom-in and zoom-out.
The remainings are used to position the target object in space.

This invokes the system routines to plan and execute a grasp towards the target. While the planning  is taking place you will be able to observe solving process. After the planning is finished to execute the planned grasp you should click to REACH button Subsequent REACH clicks will re-execute the grasp (no planning). The planning may take some time depending on your computer's performance. On a 400Mhz machine it shouldn't take more than 10seconds. If something goes wrong you can break the inverse kinematics solution process by pushing BREAK or VISREACH button again.

This button executes the planned grasp. Can be called as many times as desired. Can be used in conjunction with BELLSHAPE, PROFILE.

 This is an useful feature. After a grasp is planned this button can be hit before subsequent REACH requests to have the system to plot the three dimensional trajectory of the wrist profile for the grasp (e.g. distance between the thumb and finger end points)
Note that everything is drawn on 3d workspace so you can use mouse to look at the trajectory plot from any angle.

This button resets the camera position.

  This button resets the the robot arm/hand.

This button is used to break planning or reaching processing. If an impossible reach target position is specified it may take some (long) time for the planner to discover that the target is out of reach. In this case it is  desirable to break the process with this button.

Changes the 3D rendering mode (included modes: wireframe, skeleton (joint system), shaded, edged and shaded)

This button pops up a control window from where it is possible to control the robot arm/hand manually (via scrollbars). This way you can set up a an initial robot configuration and invoke the planner using VISREACH button. The CLOSE button in the control window closes the pop-up window.

It is sometimes useful to change the speed to a slower value (e.g. 2) to get finer trajectory and profile plots. Note that this value also controls the speed of the reach execution. Higher values (>5) may result in reach errors in spite of a correct plan .

Rotates the camera around the [xyz] axis  indefinetely until the button is hit again. Used only for earier visual        inspection of the 3D arm.

Swithced between double buffered graphics and single buffered graphics. Latter is faster former is nicer.

Switches between linear reach and realistic bellshaped profile reach. This is done by modifing an existing grasp plan (a.g. after VISREACH finiished) so that the trajectory travelled by the hand is linear. Used for experimenting the recognition capability under unrealistic grasp (in terms of kinematics) observation.

Cycles through the different objects that can be grasped (except the mug; it doesn't have the right affordance at this time). Currently each object has a single affordance. But this can be overridden by GRASPTYPE button where any object can be forced to be grasped with any grasp (Note that not all will succeed).

Swithces between the grasps allowed: SIDE, POWER, PRECISION, POWER2 (not reliable at this time),  REACHONLY (bogous at this time), NATURAL (this is means the default affordance should be  used).

This executes and planned grasp and tries to recognize it by using features of the kinematics arm. When used first time it will pop up a window where the network used to recognize will be visible. The controls in this pop-up window should be ignored at this time. The network will use a pre-trained weight set. As the grasp is being performed the decision of the network can be seen on the  information  window over the BREAK button. When the grasp is finished the final decision will be shown in the same area.  Typically the networks decision will be pretty much off during the first half and will not change too much.

1. Change the object postion using the right three scroll bars on the left panel.
2. Change the object using OBJECT
3. Change the size of the object using the second scroll bar.
4. Press VISREACH
5. If succesfull confirm (visually) using REACH button.
6. You inspect the action using the [xyz}EYE controls or using the mouse to change view angle
7. Hit RECOGNIZE to make the system recognize the grasp to be executed (the last planned grasp)

BUGS
There are some 3D display bugs arising from the bad programming practice of threads. To be fixed soon. An abnormal arm (a deformation in the arm) indicates that you hit the bug.
The inverse kinematics can not be solved all the time sometimes due to the nature of object/grasp and the position of the object (can not reach a very far target). However there might be cases where the heuristics used for grasp planning my fail. The precision grasp is the most dependable then comes power grasp. The least dependable heuristics is the one for side grasp. Improvements is underway.
The recognition depends on pre-trained network weights. The training employed tried to cover whole space of actions. However you may find regions where the training did not do very well. Especially as the number of side grasps in the training set was less than the other two, the recognition rate for side grasp can be lower (The valid side grasps that could be produced was less than the rest due to the (imperfect) heuristics used).
In Netscape to restart the applet (you should tell the browser to reload/restart the applet)  hold-down shift and press RELOAD button.
~

Erhan Oztop,  June 20, 2000
erhan@java.usc.edu