Flow field-based manipulation of objects (Credit: Microsoft Research)

Unlike many similar systems, PlayAnywhere’s projector and camera are placed on the side of the tabletop, and the device can sense and project images on a 40'' diagonal area at a relatively short distance. Andrew Wilson, one of the researchers who worked on the project, said the system demonstrates a number of important sensing capabilities that exploit the flexibility of computer vision techniques. These sensing capabilities include a novel touch detection algorithm based on the observation of shadows, and an optical flow-based algorithm for the manipulation of onscreen objects that does not rely on fragile tracking algorithms.

PlayAnywhere consists of three primary components: a projector, cameras, and an infrared illuminant – constructed as a single piece. The projector stands on a short pedestal, which is placed directly on the targeted surface, and uses four aspheric mirrors to project a visibly correct rectangular image from oblique angles at short distances. Multiple cameras and the infrared illuminant are mounted on the projector itself, in order to preserve the calibration of the vision system with the display, regardless of where PlayAnywhere is situated.  

Due to a novel approach to touch detection developed by Microsoft engineers, PlayAnywhere does not require special instrumentation of the surface on which it operates and can be used on any available flat space,. The applied technique exploits the change in appearance of shadows as objects approach the surface – for example, it can detect whether a finger is touching or hovering above the surface by analyzing the shape of its shadow. Moreover, visual codes can be used in order to distinguish between various game pieces, printed pages, media containers, knobs and other objects on the tabletop. The system can tell apart objects with similar shapes, but with different application semantics.

3D graphics model is projected onto an identified game piece (Credit: Microsoft Research)

PlayAnywhere allows easy manipulation of projected documents - users can move and rotate virtual pages as the system tracks position changes in real-time. Thanks to a special page tracking algorithm it is also possible to continuously track multiple paper pages of known dimensions, while preserving the individual images projected on each page. The users can also resize various objects shown on the tabletop, such as photos or videos, by spreading or contracting a single hand, or by moving two hands closer or further apart.

Wilson believes that the advancements being made today in sensing and display technology will boost the development of interactive display applications for home and office. Microsoft envisions such systems to be used as virtual desks that will conveniently project documents, and maybe even serve as virtual playgrounds – “Imagine a child pulling such a device out of the closet and placing it on a table or the floor of their room to transform the nearby surface into an active play space” – Wilson said.

You can find more information on Microsoft’s PlayAnywhere here.