The Present and Future

Wearables

The most prevalent and developed form of wearable device to date is the smartwatch. Ranging in sophistication from watches enhanced with step counters to the Bluetooth-paired Apple Watch capable of running standalone third-party software, smartwatches have brought a number of cutting edge interaction paradigms to the consumer market. Step counting, heart rate monitoring, and exercise tracking emphasize the quantified self. Vibration and Apple's Taptic Engine emphasize haptic interfaces and unobtrusive notifications and intimate interfaces. All in all, they are a veritable cornucopia of futuristic interaction concepts.

Model lineup of 2016 smartwatch market. Via New Atlas.

Although a market failure, Google Glass attempted a wearable interface in the eyeglasses form-factor. The device included a piece of optical acrylic to allow a small square of interface to be superimposed on the edge of the wearer's vision. Interaction was by buttons and a swipeable touchpad on the temple. Unlike the smartwatch, Glass was a standalone device and not merely a smartphone peripheral. Also unlike the wrist watch, Glass lacked the former's relatively long and rich history of added technical features (known in the industry as "complications", and commonly calendars, stop-watches, timers, and even miniature calculators). A combination of annoying early adopters (derisively referred to as "Glassholes"), unaddressed privacy concerns, and too-aggressive features doomed Glass commercially.

As recently as 2006, researchers at MIT's Media Lab described, prototyped, and tested a system which Glass very closely resembles. They coined the system eye-q, and in their paper present a salient overview of contemporary research outlining key interaction and usability goals for such systems. Of particular interest are their concerns about subtlety and intimacy for both the user and others in their surroundings. Glass suffered tremendously by ignoring the latter.

Augmented Reality

Augmented reality refers to interfaces that augment (modify) the user's perceived environment in some way. This differs from virtual reality in that the latter is a wholesale replacement of the real environment with a generated one, while augmented reality typically supplements the environment with superimposed additions. It made its pop-culture debut in early smartphone apps which used real-world orientation tags (known as fiducials) to easily track real-world locations and superimpose some virtual object into a live or real-time video view from the device's camera.

Fiducial-based augmented reality app for iPhone. Photographer unknown.

Although it is not a peripheral input device for a mobile phone, and can be called "smart glasses", Google Glass is not exactly an augmented reality device. It is in fact simply a minimally-designed optical head-mounted display. The only true augmented reality product on the market is Microsoft's HoloLens, a self-contained standalone headset device designed to project generated images seamlessly over the viewer's field of view and superimpose them into the physical environment.

Promotional image for HoloLens showing idealized vision of apps and functionality. ©2016 Microsoft.

The reality, unfortunately, is not quite so polished, as shown in these videos by Reddit user /u/drakfyre. Despite these mechanical limitations, the underlying technologies appear quite robust for a first-generation product, and he claims they're sufficient to get a sense of the paradigm's potential:

I realized how much it made me feel like a wizard when I would make a hand gesture and nothing would happen, and I forgot that I wasn't wearing it. I just expect to be able to summon a menu from my hands...

[Augmented reality is] where things are going, for certain. The ability to just create objects, ideas, windows, screens, even simple devices like dice and gameboards; the ability to measure your space and visualize what could be, the ability to create virtual people, avatars, and creatures, and all of this networked so that anything you create others can see. You'll put a big screen in the middle of your lunch table and share the latest YouTube videos. This is stuff I have access to currently, but there's a lot of usability things that could be better. That's why I am excited; I like to see the iteration. [...] I want better hand and finger tracking and wider FoV. But what I have is incredible... what I want more than anything else is for more people to be able to do what I can do now. And see what other people do with it.

Virtual Reality

The limitations of physically-based interactions are currently a major roadblock in the development and adoption of virtual reality systems as well as augmented reality systems. However, while physical limitations of augmented reality are generally restricted to small scale objects and fine motor manipulation (sub-meter scale objects), virtual reality presents challenges of physical credibility from the sub-meter through the kilometer interaction scale. This is because in virtual reality, the entire shape and form of the environment are supplanted (including whole buildings) whereas in augmented reality generally only the contents of the environment or its appearance are modified.

These problems with the physicality of VR environments are already having frustrating side effects. Almost everyone who has used one of today's commercial VR systems can describe collisions with real-world obstructions that they have experienced first or second hand. Sometimes, though, virtual objects without a physical counterpart are at fault. This GIF of a small child falling over is a prototypical example of the dangers of current VR systems.

Comparison image showing HTC Vive and Oculus Rift controllers. Source.

Thus far consumer VR systems, such as HTC Vive and Oculus Rift, all rely on custom input controllers for the operator to use within a fixed interaction arena. Since these products are tethered by cable to a sufficiently powerful computer system, the user is only cable to walk around in a space a few meters on each side. Locomotion within the virtual world most thus be restricted to a similarly sized area, or be accomplished through some trickery, else the wearer run into the walls of their room.