A Domain-Independent System for Sketch Recognition

Bo Yu and Shijie Cai

Summary

The authors present a domain-independent sketch recognition system. The system has two stages: an imprecise stroke approximation stage, and a post-processing stage. In the first stage, curvature and direction is calculated for each point. Area is used to direct recognition and verify results. Feature area for a stroke is determined by the area between points on a stroke and another reference object.

The system combines a corner finding approach with a primitive shape approximation approach. First, a check is made to see if any the stroke can be approximated by any primitive shape. If it can, the process stops; otherwise, it divides the stroke at the highest curvature point, and continues the process on the two new segments.

Line segments are approximated using least-squares best fit line to check if derivation is small between the stroke and the best fit line. If it is, the feature area is used to validate the candidate line. Curve segments are approximated by checking if the direction graph can be approximated as a line. Feature area is once again used to validate candidate curves.

In the post-process stage which occurs after the sketch has been complete drawn, false and redundant objects are removed, strokes are beautified, and domain-independent objects are recognized.

The system has a user interface to help users obtain their desired drawings. A modal toolbox exists for the creation, modification, and deletion of primitive shapes, as well as, command gestures for copy, delete, undo, and redo.

The authors conducted an evaluation where the system achieved accuracy for primitive shapes and polylines was 98%. The system obtained an accuracy of 70% for hybrid shapes with smooth connections.

Discussion

The incorporation of corner detection with primitive approximation seems beneficial. Corner finding algorithms are still not perfect. Any added assistance to help in the segmentation of strokes is useful. The use of command gestures was nice; however, having to use the modal toolbox to enter modification mode is not so nice. I understand the complexity of distinguishing gesture commands from actual sketching. An interesting piece of future work would be to investigate ways to separate these two. I could see using FlowMenus for this system, instead of a modal toolbox.