A Brief History Of Augmented Reality

In 1994, the term mixed reality appeared, which is the term Microsoft uses today. On one hand there is the real world and on the other hand there is the virtual environment. And in between there is a continuum which is mixed reality.

Then in 1997, Ronald Azuma provided a widely acknowledged definition for AR as identified by three characteristics: it combines real and virtual, it is interactive in real time and it is registered in 3D.

Ivan Sutherland created the first AR system

DirectIndustry magazine: So AR technology is very old. Why didn’t it take off in the 90s then?

Michael Gervautz: All these technologies and systems were built with huge silicon-graphics machines with a lot of cables between the machines and the head-mounted system. They were not mobile, so it was impossible to move them around. Users were contained in one place, so this explains why this technology was not brought to the commercial market. In 1997, Steven Feiner presented the Touring Machine, the first mobile augmented reality system (MARS). It uses a head-worn display you look through with an integral orientation tracker; a backpack holding a computer, differential GPS, a digital radio for wireless web access; and a hand-held computer with stylus and touchpad interface.

Steven Feiner presented the Touring Machine

DirectIndustry magazine: What potential did you envision for this technology at that time?

Michael Gervautz: At first, we were betting on marketing, sales and entertainment markets. In 2006, one of the first advertisements for the WWF in China used Augmented Reality. And slowly, we realized there was huge added value in deploying AR technology in the enterprise. But it took a while to go from a technology coined for entertainment to industrial applications.

DirectIndustry magazine: When did people start to see the potential?

Michael Gervautz: Digital transformation was a huge trigger for AR. Around 2013, going digital became the new industrial trend. Having a digital representation of your product throughout its lifecycle became essential and AR can play a role here. AR can help with remote assistance, 3D instructions and design engineering. All of a sudden, AR was becoming important. But there are still some blockers.

DirectIndustry magazine: What are those barriers that need to be overcome for there to be AR in the enterprise?

Michael Gervautz: You need to build the digital twin and digital thread to leverage all the digital data. So as long as you are not able to use digital data throughout your chain, you will not be able to see AR in an effective way. There are also challenges to creating a good AR user experience and having hands-free devices. And the recognition and tracking of products and environments is also a challenge.There are still products that computer vision and AR systems cannot recognize. There are still things to do but we are making progress.

“AR can help with remote assistance, 3D instructions and design engineering. All of a sudden, AR was becoming important.” (Vuforia Expert Capture used in industry / Credit: PTC)

DirectIndustry magazine: What are the advancements you did not expect 20 years ago?

Michael Gervautz: Machine learning and deep learning are influencing computer vision in a way we had not expected. Before, you needed people with a PhD to understand AR which was a complicated technology. Today, with machine learning, you basically throw images to a neural network, tell it what it is, and train it to recognize what it sees. That was one of the surprising things.

DirectIndustry magazine:  And what about haptic devices?

Michael Gervautz: If you want to have a full AR experience, you might want to have all the senses like touch or even smell as well. But in most cases, haptic does not play an important role. You can do a lot of things without haptics. In 2 years, we might have AR systems that are able to recognize a large variety of objects. We could then be able to cover 95% of the use cases without haptics.

DirectIndustry magazine: What are the next challenges?

Michael Gervautz: Indoor navigation, what people call spatial computing, is the next step. Spatial computing is a new form of computing that uses computer vision and artificial intelligence to seamlessly blend content into the real world. It’s a technology that allows you to recognize the environment, the spaces where you are. We are able to augment objects and products. Soon, we will be able to augment and track the environment. We will build a digital twin of the environment by using a 3D scan and thus create navigation applications and spatial instructions using the surroundings as interactive elements to be explored.That will be a huge step forward. It will be possible, for example in a factory, to lead a technician to a faulty machine via the shortest path. PTC has just released a technology called Area Targets that delivers augmentations to stationary objects in a scanned space. For the moment, it is limited to 2,000 square meters but we are working on increasing the size of the areas.

DirectIndustry magazine: So, in a nutshell, how has AR changed industry?

Michael Gervautz: AR is able to answer spatial questions in a visual way. It can point you to a place or an object. AR is the new interface into our world. It’s changing the workplace by making the work easier for the workers. AR technology is also making industry more attractive.