LADDER, a sketching language for user interface developers

Tracy Hammond and Randall Davis

Comments

Nabeel's blog

Summary

The authors present a sketching language for describing recognizable shapes and sketches to be used by persons not in the sketch recognition field. The goal is to allow people such as user interface designers and developers and experts in a specific domain to easily build systems that use sketch recognition interfaces without having to build a complex sketch recognition system from the ground up.

The described language, LADDER, defines recognized shapes using five parts: components, geometric constraints, aliases, editing behaviors, and display methods. Components are the elements that make up a shape. Constraints define the relationships between the components. Aliases specify names for components that allow for easier readability and construction in the language. Editing behaviors describe how a shape can be modified. Display methods specify how a shape is visually shown. Shapes can be defined in a hierarchy. As well, abstract shapes can be defined to describe general features shared by multiple shapes to prevent redundant declarations.

Constraints for a shape can either be specified with rotationally invariant constraints or to a specific orientation. A shape can be labeled as isRotatable when constraints are described with respect to orientation, but any orientation is acceptable.

Editing behaviors are broken down into actions (how a shape is modified) and triggers (how this action is executed). Display methods allow the shape to be drawn in a number of different ways including unchanged or cleaned-up.

The recognition system first tries to recognition primitive shapes. If only one possible primitive shape can be assigned to a drawn shape, it is passed on to the next stage; otherwise, all potential classifications for the drawn shape are passed on. The next stage recognizes domain specific shapes using a rule-based system. Each recognizable shape has a rule. The domain recognizer searches for every possible combination of primitive shapes that can satisfy a rule.

Code for domain shape recognizers, display methods, and editing behaviors are generated in a translation stage run prior to run-time.

Discussion

The authors have contributed a sketching language for non-experts to build systems that require sketch recognition. This allows experts of a domain design sketching systems applicable to their work. This minimizes the workload on the sketch recognition community to learn domain specific knowledge for building domain specific sketch recognition systems.

One thing I'm interested about is if the language is extensible. By extensible, I mean users can define constraints, actions, triggers, etc. beyond the predefined ones. This seems particularly applicable with constraints where they involve mathematical equations. The equation for the constraint could be stated in the language, and no new programming would be involved for the end user (just more use of the language).