Are We Heading Towards a Screenless Future?

Interface advances are set to blur the line between the real world and the digital one. While new interfaces such as AR glasses and hand-free headsets are being developed, the question is being raised whether screens, which are so predominant now,  are set to disappear.

It is nearly 10 years since the iPad first hit the market. Along with the iPhone, it was among the first devices to popularize portable touchscreen interfaces. But since then, not much has changed in computer interface design, and the majority of computer interaction still revolves around the QWERTY keyboard, first devised in the 1870s.

But all this is gradually changing. Introduced in October 2019, Google’s Pixel 4 phone allows users to control the device with gestures via on-board sensors. Meanwhile, developments in augmented reality, glasses, lenses and interactive projection are also advancing. Research by Goldman Sachs has found that the global market for augmented and virtual reality is expected to grow by over 50% every year, reaching $95 billion by 2025.

The Future is in Augmented Reality

AR places an information layer on top of the real world the user sees, usually via a smartphone, but headsets, glasses and even contact lenses are in development to support the interface.

Rather than a shift to a new mode of computer-human interactions, developers need to accept the idea that the future is multi-model: that users will switch between keyboards, touchscreens, voice, gesture and AR, according to Tuong Nguyen, senior principal analyst at Gartner:

“If you think back to when the computing era started, someone had to figure out how humans were going to interact with the digital realm and they ended up with the mouse and keyboard paradigm. Then we moved on to smartphones and tablets, and the keyboard doesn’t quite work in that realm, so someone figured touch interfaces. The next era we’re moving into is the marrying of physical and digital in augmented reality.”

The tricky part is AR is not a single technology, in and of itself. It puts together  different technologies in a certain way that creates an experience that you will know as AR. Nguyen continues:

“It will be a mix of interfaces that we currently have together with the next-generation human-machine interface. So, we will not just be typing on keyboards, but using motion, gesture, and voice. It will be multimodal.”

Microsoft’s HoloLens and Facebook’s Oculus headsets, along with others, are popularizing AR for gaming and other consumer applications. An elementary form of AR already hit the mass-market with Niantic’s Pokemon Go smash-hit mobile game.

In this video interview we did in 2018, Nancy Li, from the AI perception department at Microsoft explains how Hololens is driving digital transformation in companies. She also believes that in the future, we may all wear smart glasses before moving to smart contact lenses.

“There are technologies patterns out there that are already in that phase, looking at things like that.”

The State of Industrial Augmented Reality

Industry is also keen to reap the benefits of AR. According to research from industrial software supplier PTC, 24% of enterprises experimenting with AR have done so with the aid of “new” devices including headsets allowing hands-free work. B-netween 62% and 55% of respondents are making experiences available on smartphones or tablets. The 2019 study found 68% of organizations feel confident that they will move their AR initiatives to production within 12 months.

While some headsets allow users to view augmented reality, others let them interact with it, for example, AR hardware company Magic Leap has shipped a headset supporting cameras that allow users limited interaction with the virtual object they see in front of them. Meanwhile, projected interactive displays can turn almost any surface into a touchscreen. Mixed with object recognition, they allow the system to add smart features to real-world objects, from the user’s perspective.

Mark Rolston is the founder and chief creative officer of Argodesign. He came up with a technology, Interactive Light, that allows users to interact with virtual content without necessarily having a computer in front of him or her or without wearing a headset. For him, such a system will be able to “lock” information to real objects for when it is needed.

“If I’m working on a jet engine, I can set some information to float right next to a specific component, and it will stay right there, even as I turn my head, and maybe go to the bathroom, but the instructions are still over there attached to the object.”

Watch Argodesign video

New Interfaces

Others are looking to let computer interaction slide into the background. French firm Woodoo has developed a treatment for natural woods which removes the lignin and refills the material with a specially developed polymer, which also fills the gaps between wood fibres. In addition to making the material extremely strong and fire-resistant, it can also be translucent and turned into a touch display. The effect is a display hidden as a wooden panel or tabletop, which comes to life when the user requires, according to Olivier Grange, senior vice president marketing communications. The company is already working with Daimler and Airbus to integrate displays into cars and aircraft.

The technology industry has long fantasized about giving users the immersive, ever-present and discrete user interfaces seen in films like Tom Cruise’s Minority Report. Over the next five years, we’ll see if this will become reality.