Gestures without Libraries, Toolkits, or Training: A $1 Recognizer for User Interface Prototypes

Jacob O. Wobbrock, Andrew D. Wilson, and Yang Li

Comments

Manoj's blog

Summary

Wobbrock, et al., introduce a new light-weight gesture recognizer that doesn't require training or special libraries and toolkits. The algorithm was designed to work on any development platform and be simple enough for UI designers to create gestures to be recognized by it. The algorithm has four steps.

1. The point path is re-sampled so that points on the path are equidistant from each other. This is needed so that speed a gesture is drawn has no effect on the gesture recognition.
   
2. Rotate the gesture once based on the indicative angle.
3. Scale the gesture to a reference square. Then, translate the gesture, so it's centroid is at the origin.
4. Find the optimal angle for comparing the gesture with the templates to obtain the best score.

The authors compared their algorithm with both Rubine's and a template matcher based on Dynamic Time Warping (DTW). The results showed that their algorithm performed with better accuracy than Rubine's, and comparable with DTW. Also, the authors noted the $1 recognizer had a greater separation between the 1st and 2nd gesture scores.

Discussion

The contribution of this work is a simple gesture recognizer that does not require extra software, training, or expert knowledge in the field of pattern recognition. The authors developed two implementation of the $1 recognizer, one in C# and one in javascript. It's ability to be implemented in a variety of platforms including light-weight ones such as javascript and Flash seems incredibly valuable.

To handle variability in gestures, the authors use an idea of aliasing, so that multiple templates can be assigned to one visual object (e.g. arrow example, see Figure 7 from paper). The simpleness of the algorithm probably limits them to this approach, but it seems future work could be directed towards addressing this problem better.

At interesting point to me is that in the evaluation the authors asked participants to rate the gestures subjectively. It surprises me that this has not been done in other work we have looked at so far. It seems to me that getting input from actual users about the quality of gestures is an important step in choosing gestures to recognize. Of course, I realize the majority of the work is about the actual recognition of the gestures and not the gestures themselves.