Visual Similarity of Pen Gestures

A. Chris Long, Jr., James A. Landay, Lawrence A. Rowe, and Joseph Michiels

Comments

Nabeel's blog

Summary

The authors propose metrics for evaluating the visual similarity between pen gestures. The limits of human attention and memory make it difficult to recall gestures. Their goal is to develop a systematic method for checking the similarity between gestures, so that gestures can be designed different enough from each other to not confuse the user during recall.

Two experiments were conducted by the authors. For each experiment, a set of gestures was defined. For the first experiment, the authors wanted to derive metrics for evaluating the similarity between gestures, so they generated a set of gestures that covered a wide variety and that reflected different spatial orientations. In the first experiment, participants were shown all possible combinations of creating three different gestures from the set and asked to pick the least similar gesture of the three.

The authors selected several possible features for comparing similarity. They looked at the 11 features used by Rubine, as well as others derived from examination of the data using a technique called multi-dimensional scaling (MDS). Some of the predictors resulting from the first experiment where curviness, total absolute angle, density, cosine of angle between first and last points, and aspect.

In the second experiment, the authors wanted to evaluate the predictive power of the metrics derived from the first experiment against new people and gestures, and explore how changing different types of features would affect the results. A new gesture set was derived for the experiment based on these criteria. Some of the resulting predictors for the second experiment were Log(aspect), total absolute angle, and two density metrics.

The authors found that the predictors from experiment 1 were better at predicting than those from experiment 2. Also, the authors noticed in both experiments that participants used different features in determining similarity.

Discussion

This work introduces metrics for evaluating the similarity between pen gestures. Developing systematic ways to analyze the similarity between gestures allows developers to design gestures that are dissimilar for commands that are unrelated, and similar gestures for those commands that are related.

The two faults I find with this work are one, as mentioned by the authors, they did not let the participants actually draw the gestures. Unforeseen changes in perception may arise when the participants are engaged in drawing the gestures. The other fault is that I did not believe the authors adequately examined research in perception related to this topic. There has been work done on perception of similarity by psychologist since the beginning of the early 20th century. Spatial proximity plays a role in how we perceive similarity. The snapshots of the study show the three gestures close to each other. This could have adverse effects on their results.

I think this type of research is incredibly valuable to sketch recognition. My future work would be to run more studies with the issues previously discussed addressed. A continued refinement of the metrics is needed, which can only come with further evaluation and resulting insight.