Paper Based Interfaces

People love using paper. Despite the growth of electronic documents, our use of paper has increased. People use paper for taking notes, for drawing figures, for thinking, for organizing thoughts. What if we could capture these paper documents and connect them to the digital world? This was the main impetus behind the project titled "Paper Based Interfaces" that I worked on at Intel Labs.

The idea behind the project was discovering a way to implicitly tag handwritten documents. We assumed that these documents were captured by a camera or scanner of some sort, and that we had, as our raw material an image. We wished to form a digital representation of the document that could be stored, retrieved, indexed and linked to other digital information. We used segmentation and transformation to extract features from the document, creating a set of document keys.

We allowed the user to draw command glyphs on the document that could, for example, cause the document to automatically be e-mailed to some address or uploaded to a blog. We allowed users to underline keywords that could be recognized and attached to the document as content descriptors. The segments in the document and their geometric relationships were used to implicitly tag each document. Given a new scan of an old document, the old document could be found in the database by using these features.

The demo for the project was developed using MFC to create a GUI front end and Access as the backend database. This demo won "Best of Show" at a lab-wide demo fair. The work was later patented.