Graphical Input through Machine Recognition of Sketches

Christopher F. Herot

Comments

Manoj's blog

Summary

Herot presents an approach to sketch recognition that infers user intent by measuring how quickly a pen is moved and how hard it is pressed. He introduces a system called HUNCH and discusses several sketch processing features within it (e.g. latching and overtracing), including problems faced by those features. HUNCH is able to map 2D sketches into 3D structures. The author points out that human understanding of context helps the user make sense of a drawing, and computers could benefit from having a similar knowledge structure. He details the HUNCH system's approach in which context is specified by the user.

The final designed system is a more interactive system that incorporates improvements to the features of HUNCH. The system is able to recognize lines and curves on the fly based on the speed of the stroke and it's "bentness." In HUNCH, the user specified when the processing features were run. In the new system, these are ran in the background to avoid disrupting the user's flow in designing.

Discussion

The author has designed graphical input system for recognizing sketches that seeks to use both machine processing of sketch features and human understanding of context. I think it's better to ask user about context, than to wrongly determine context. However, persistent querying of the user also seems wrong. An adequate balance is needed. Questions about context should be asked "in-context", that is appearing at the point of interaction. Forcing the user to look elsewhere on the screen defeats any benefit from asking him/her for feedback. This of course means developing clear transitory affordances for the user to interact with the system. It sounds as if the author was headed on this approach, but I am uncertain how exactly the user provided contextual information within the presented system.