Skinput

The Skinput system rendering a series of buttons on the arm. Users can press the buttons directly, with their fingers, much like a touch screen.

Skinput is an input technology that uses bio-acoustic sensing to localize finger taps on the skin. When augmented with a pico-projector, the device can provide a direct manipulation, graphical user interface on the body. The technology was developed by Chris Harrison, Desney Tan, and Dan Morris, at Microsoft Research's Computational User Experiences Group.^[1] Skinput represents one way to decouple input from electronic devices with the aim of allowing devices to become smaller without simultaneously shrinking the surface area on which input can be performed. While other systems, like SixthSense have attempted this with computer vision, Skinput employs acoustics, which take advantage of the human body's natural sound conductive properties (e.g., bone conduction).^[2] This allows the body to be annexed as an input surface without the need for the skin to be invasively instrumented with sensors, tracking markers, or other items.

Microsoft has not commented on the future of the projects, other than it is under active development. It has been reported this may not appear in commercial devices for at least 2 years.^[3]

Operation

Ten channels of acoustic data generated by three finger taps on the forearm, followed by three taps on the wrist. The exponential average of the channels is shown in red. Segmented input windows are highlighted in green. Note how different sensing elements are actuated by the two locations.

Skinput has been publicly demonstrated as an armband, which sits on the biceps. This prototype contains ten small cantilevered Piezo elements configured to be highly resonant, sensitive to frequencies between 25 and 78 Hz.^[4] This configuration acts like a mechanical Fast Fourier transform and provides extreme out-of-band noise suppression, allowing the system to function even while the user is in motion. From the upper arm, the sensors can localize finger taps provided to any part of the arm, all the way down to the finger tips, with accuracies in excess of 90% (as high as 96% for five input locations).^[5] Classification is driven by a support vector machine using a series of time-independent acoustic features that act like a fingerprint. Like speech recognition systems, the Skinput recognition engine must be trained on the "sound" of each input location before use. After training, locations can be bound to interactive functions, such as pause/play song, increase/decrease music volume, speed dial, and menu navigation.

With the addition of a pico-projector to the armband, Skinput allows users to interact with a graphical user interface displayed directly on the skin. This enables several interaction modalities, including button-based hierarchical navigation, list-based sliding navigation (similar to an iPod/SmartPhone/MID), text/number entry (e.g., telephone number keypad), and gaming (e.g., Tetris, Frogger)^[6][7]

Demonstrations

Despite being a Microsoft Research internal project, Skinput has been demonstrated publicly several times. The first public appearance was at Microsoft's TechFest 2010, where the recognition model was trained live on stage, during the presentation, followed by an interactive walkthrough of a simple mobile application with four modes: music player, email inbox, Tetris, and voice mail.^[8] A similar live demo was given at the ACM CHI 2010 conference, where the academic paper received a "Best Paper" award. Attendees were allowed to try the system. Numerous media outlets have covered the technology,^{[9][10][11][12][13]} with several featuring live demos.^[14]

References

External links

• ACM Digital Library - Skinput: appropriating the body as an input surface