Multimodal Collaborative Handwriting Training for Visually-Impaired People

Beryl Plimmer, Andrew Crossan, Stephen A. Brewster, and Rachel Blagojevic

Summary

The authors present McSig a learning system for teaching visually-impaired students handwriting and how to sign their names. The system consists of a force-feedback device that restricts movement of a pen while learning. The force-feedback is gradually reduced as a student becomes more familiar with how a shape is drawn. A teacher is able convey how something is drawn by drawing it on a separate screen, and the force-feedback device replicates the drawing movement. They conducted an evaluation consisting of 8 visually-impaired students over the age of 10 and still in school. The evaluation highlighted design issues and provided indication that this system could possibly help the visually-impaired learn handwriting.

Discussion

The work is interesting. It's always great to see work applying technology to help make lives easire for a those in need, even when they are only a small subset of the population. The force-feedback echoing when the teacher draws a shape seems like a nice and helpful idea for teaching. I wonder if a device such as this could be used to improve motor skills of those you have injured their hands and need physical therapy.

Sketch Recogniton User Interfaces: Guidelines for Design and Development

Christine Alvarado

Summary

The author introduces SkRUIs, sketch recognition user interfaces, as new type of interface not addressed in previous literature. Prior work has focused on HCI for pen-based input or sketch recognition, but not the combination of the two. An example application for drawing diagrams for a power presentation is presented. The author states that traditional HCI evaluation techniques are not entirely suited for SkRUIs, and these techniques require modification to support SkRUIs.

Discussion

While it's nice to see research being done in this area, I'm not convinced by the work. Why can't Powerpoint support incorporation of sketch recognition for diagramming? Why must it be done in a separate application? Doing beautification on window switches doesn't seem like the best idea. What if I'm drawing, but I get an instant message in the middle and decide to check it. I may not be ready for beautification to occur. A SkRUI is still a GUI, just more specific. The interaction fundamentals of GUI design still apply.

Fluid Sketches: Continuous Recognition and Morphing of Simple Hand-Drawn Shapes

James Avro and Kevin Novins

Summary

The authors introduce a new form of visual feedback about sketch recognition. Feedback is provided as shapes are drawn. The approach beautifies portions of the current stroke to reflect the recognition systems understanding of what is being drawn by the user. The approach works for two shapes, circles and squares.

Discussion

The approach is novel and interesting. There are two main problems I find with this work. One, it only works with two shapes. Two, they don't evaluate the effects of the feedback on human attention. Does it disrupt drawing? Do users find themselves waiting for feedback before drawing too far into a stroke?

Interactive Learning of Structural Shape Descriptions from Automatically Generated Near-miss Examples

Tracy Hammond and Randall Davis

Summary

Hammond and Davis present an approach to fix over- and under-constrained shape definitions for a shape description language. The authors developed the approach for the LADDER shape language. Their approach requires a positive hand-drawn example and shape description that will properly recognize the provided example. Over-constrained descriptions are checked by sequentially negating each constraint and constructing a near-miss shape that tests the constraint. Under-constrained descriptions are checked similarly, except that negated constraints are added instead of checking existing constraints.

Discussion

Having used LADDER, the ability to uncover under- and over-constrained descriptions is incredibly valuable. It's very easy in LADDER to under-constrain a description.

What Are Intelligence? And Why?

Randall Davis

Summary

Randall Davis presents "definitions" for intelligence and why intelligence exists. The definitions of intelligence come from the five areas of study looked at by artificial intelligence researchers: mathematics (logic), psychology, biology, statistics, and economics. Logicists view intelligence through formal calculations of logical rules. Psychologists view intelligence as human behavior. In biology, intelligence is viewed as a response-stimuli behavior based on the physiological architecture. Statistics provides a probability theory approach, and economics presents an approach based on utility theory.

Davis proceeds to explain possible reasons why intelligence exists through an exploration of how the human mind evolved. Fossil records show that the encephalization quotient (ratio of brain size to body size) of human ancestors began to increase over four million years ago. However, early man did not begin developing tools and language skills until 300,000 years ago. He points out a number of theories why this may be. He also notes that evolution is more of random search than a goal-oriented process, and that the products of evolution are often messy and multifaceted.

A number of examples of animal intelligence are presented. These examples serve as to point out difference in intelligence, while also showing similarity between animal intelligence and human intelligence. The goal being to find ways to uncover aspects of human intelligence by investigating simpler forms of intelligence in animals.

He concludes the paper with an exploration of the idea that we think by "reliving." He explains evidence for how we create concrete visual ideas in our mind. In order to answer questions of what would happen, we picture in our mind visually how something would play out to answer these questions.

Discussion

I enjoyed this reading. It contains a lot of interesting information about areas that I know only a small amount about. I found it very captivating to look at how different areas look at human intelligence, and to theorize how it came about through evolution-based analysis. My only questions are:

• How can we apply these different views of intelligence?
• How can we integrate the views?
• Are certain AI tasks better suited for specific models of intelligence? In other words, is the one view or combination of views that would work best to address a specific focus topic within artificial intelligence?

The view of human intelligence formed from a messy layering of evolutionary forces is a nice take that makes sense to me. The complexity of human intelligence comes not only from how advanced it is, but also how complicated and inefficiently designed it is. It makes me think that looking at simpler animal intelligence is a good idea for building a basis for looking at human intelligence.

Magic Paper: Sketch-Understanding Research

Randall Davis

Summary

This paper presents an overview of the field of sketch understanding and more specifically a sketch understanding system called Magic Paper. It points out reasons for developing systems that understand free hand sketches. A number of problems with sketch understanding are described. Solutions from other areas such as speech recognition are pointed out and explained why they are not suitable for sketch recognition. The basics of sketch understanding are presented including sketch representation, finding primitives, and recognizing shapes. A number of sketch-enabled interfaces are described where sketch understanding is connected to a back-end system such as RationalRose or ChemDraw. Techniques for automated learning of new sketch domains and the difficulties associated with this task are presented.

Discussion

This paper presents a nice overview of the issues of sketch understanding, and why solving these issues is so challenging. The goal of this work is to create "magic paper" which affords the same natural and easy interaction as paper, but is capable of understanding what is drawn on the paper. It would seem advances in both hardware technology and software algorithms are still needed to achieve this ambition goal, but several big steps have already been taken to get us there. The concept of true "magic paper" seems the killer app for sketch understanding. Only when "magic paper" is better than or at least comparable to real paper will sketch understanding find itself deeply-seated in the daily lives of humans.

Perceptually Supported Image Editing of Text and Graphics

Eric Saund, David Fleet, Daniel Larner, and James Mahoney

Summary

Presented in this paper is an image editing program called ScanScribe. ScanScribe provides special functionality for selecting and structuring groups. Grouping is represented by a lattice structure where an image object can belong to more than one group. ScanScribe has an image analysis technique for seperating foreground and background. ScanScribe uses auomatic structure recognition to group elements of an image. The group recognition is based on Gestalt laws of human visual perception. No formal evaluation was conducted, but a number of users reported that the system was easy to learn to use.

Discussion

Our ability as humans to easily differentiate text from shape and form groupings of these different objects seems a valuable place to begin investigating for methods to employ with machines. I really like the idea of using laws of perception as a basis for building mathematical calcualtions of similarity. However, the human eye and mind do not function the same way as a computer; therefore, these laws of perception may not translate as easily to machines. As well, other factors, such as domain and contextual knowledge, play a role in our ability to differentiate shape from text.