Augmented reality technology has the power to boost front-line worker productivity by as much as 50% and reduce human errors by up to a whopping 90%. But successful applications require quality content and project prioritization.
By Michael E. Porter and James E. Heppelmann
The coronavirus crisis of 2020 is changing the way we work, , perhaps forever. Travel bans, lockdowns, and social distancing policies disrupted business around the world, forcing companies to find new ways to work. The trend toward a more distributed, mobile, and agile workforce was already unmistakable, but the crisis has dramatically accelerated this shift for workers with the skill to operate in the virtual world. But perhaps the bigger news coming out of the crisis is that the larger front-line workforce operating in the physical world participated for the first time in digital transformation in a meaningful way.
Knowledge workers in roles such as management, finance, marketing, scheduling, or product development trade in information that is readily captured in digital files and databases. Their work is easily exchanged via emails. Virtual meetings to collaborate with peers around the globe are a few clicks away. Where the work is digital, worker location doesn’t really matter. So when knowledge workers around the globe were forced into a massive work-from-home experiment with almost no planning, the results proved better than we ever thought possible. Not only did productive work continue, but additional time and cost savings were realized from reduced commuting and business travel.
These results make it clear that the work-from-home Genie probably won’t go back into the bottle, even when the crisis passes. For knowledge workers, this way of working will become part of the “new normal” because of its inherent advantages.
How Front-Line Work Differs
Not all workers spend their day behind a desk , using IT tools to engage the virtual world on their computers. Yet, approximately 75% of the global workforce consists of front-line workers whose job is done in the real world, and involves physical work.
While knowledge workers transitioned easily to work-from-home, front-line workers cannot do their work from home. In fields like retail and hospitality, many were furloughed because their businesses were closed, sometimes by mandate. Work at factories, farms, repair shops, distribution centers, and other types of physical worksites, however, was deemed essential to the economy. The Brookings Institution estimates that between 49 million and 62 million front-line workers in the U.S. alone were asked to continue reporting to work.
“AR enables front-line worker solutions that parallel the well-established digital technologies that came to the rescue for knowledge workers.”
Front-line workers also need knowledge in order to do their work. But rather than electronic files and video calls, this traditionally comes in the form of paper documentation, training classes, over-the-shoulder mentoring, and face-to-face troubleshooting. None of these modalities work that well in the best of circumstances. But they have been seriously impaired in the era of many working at home, travel bans, and social distancing. In addition to potential exposure to health risks, the coronavirus exacerbated the already serious problem for many companies of an increasing shortage of trained front-line workers.
Bringing Digital Technology to the Front Line
Office productivity tools like Microsoft Office help knowledge workers capture and share information, and videoconferencing tools like Zoom or WebEx work great for collaboration when there is a knowledge worker using a computer on each end of the information exchange. But can we facilitate collaboration between an expert knowledge worker such as a manufacturing engineer and a front-line worker such as an assembly worker out in the factory? This traditionally involved the engineer travelling to the factory, but suddenly that was not possible. And can we facilitate collaboration between two front-line workers where one is a veteran technician who needs to explain a laboratory process to a new-hire technician? Traditionally, this takes place through mentoring or job shadowing, but social distancing makes that difficult.
The answer to how to enable collaboration and knowledge transfer with front-line workers is augmented reality (AR), an emerging digital communication technology that can support and significantly enhance the productivity of physical work throughout the economy. We have discussed the concept and impact of AR technologies in our earlier Harvard Business Review article titled “A Manager’s Guide to Augmented Reality”. In the simplest terms, AR allows actual, relevant digital information to be transmitted and displayed in the context in which it is used in the physical world. This creates a powerful force for productivity improvement in a series of areas.
AR has allowed Toyota to save time and money by helping experts eliminate an average of four trips per month to production plants.
AR: Zoom for Front-Line Workers
AR enables front-line worker solutions that parallel the well-established digital technologies that came to the rescue for knowledge workers. Instead of a Zoom video conferencing call, AR technology allows remote experts to see the physical world in video and annotate physical objects during the call. Rather than publishing a PDF document or a Web page, AR can map instructional content directly onto the 3D physical environment in which the work takes place. Instead of a how-to YouTube video playing on a computer screen, AR captures the best front-line expert using a wearable device that captures every step, and creates an interactive and step-by-step guide mapped onto the work environment that other workers can follow using a wearable device.
The critical difference of AR is its power to deliver needed digital content and expert guidance into the context of the physical environment, where front-line work is actually done. This substantially reduces the cognitive distance that gets in the way of the worker translating the information from digital to the real world. Reducing this gap increases worker productivity and reduces errors.
Expert Collaboration and Remote Support
AR is a powerful real-time collaboration tool between a remote expert and the front- line worker who needs input. The front-line worker can use an AR app on a standard mobile phone or tablet to request help from an expert at another work site, at a corporate office, or even at home. The remote expert can see, and digitally annotate, the on-site worker’s physical work environment using a finger pen or a mouse, attaching these annotations to the physical objects and moving as the worker moves.
AR does for front-line workers what video conferencing does for knowledge workers – it allows expertise to be shared electronically. But unlike video conferencing, which is solely digital and on flat screens, AR has one foot in the digital world and the other foot in the physical world – effectively serving as a bridge between them. This application of AR is akin to sportscaster John Madden’s famous “chalk talks,” where he diagrammed NFL football plays on the TV screen for viewers. With AR, a real-time chalk talk by an expert is overlaid onto workers’ remote work environment.
AR Markups Overlaid on a Video Call. This factory worker is being guided through a machinery troubleshooting process using AR markups by a remote expert, overlaid on the worker’s physical environment during a video call.
The use of AR technology for collaboration and remote support of front-line workers has soared, enabling experts to be much more productive in helping to debug problems and resolve production issues remotely. Toyota, for example, has used AR video calls to save time and money by helping experts eliminate an average of four trips per month to production plants because they can oversee plant modification tasks and monitor local worker safety remotely. Using an AR-powered tool, both employees and subcontractors are able to tackle complex or unfamiliar challenges with the help of an expert where and when they need it.
Worker Training and Learning
AR is also a breakthrough technology for training front-line workers, It enables next-generation instructional content such as training materials and step-by-step or standard operating procedures (SOP) for work. AR can deliver pre-prepared training content (and avoid the need for real-time collaboration and remote support) by mapping the content directly onto the 3D physical environment in which the work will take place. This results in a step-function improvement in comprehension and retention versus traditional PDF or Web-based documents that explain procedures on pages and via text in a more abstract 2D form. AR creates a 3D web, where instructional content is delivered via a 3D experience that mirrors an object or workspace, rather than on the traditional flat web page.
Augmented Reality in the Industrial World Industrial machinery in the physical world is decorated with an AR digital display showing the status and key operating parameters of the oil system, gearbox, air system, and main motor.
To access work instructions, a front-line worker launches a web browser app on smart glasses, a smart phone, or a tablet. AI-based computer vision technology identifies the real-world objects or workspaces involved using the AR device’s video camera, and delivers the relevant content to the front-line worker superimposed on the appropriate object or workspace in the worker’s physical environment. The AR device can see the real world the way the worker does, while projecting digital information onto that world to guide a worker in performing tasks, such as a repair procedure.
A striking example is Volvo’s use of AR work instructions to dramatically improve the productivity and accuracy of final quality checks on newly manufactured truck engines. The AR application provides visual guidance on where the Quality Assurance inspector should stand relative to the engine, what part of the engine she should look at first to inspect, and so on. The AR app shows what the inspector should expect to see and compares it to the actual parts. This is repeated for each step in the quality control process, capturing in the process records of any non-conformances.
This example also illustrates the major productivity impact of AR. The training time of Volvo QA inspectors has been reduced by 60%, or about three weeks per inspector. By decorating physical objects in the actual work environment and locations with digital information, the cognitive distance problem that stands in the way of all training—trying to absorb information in abstract 2D paper form and translate it to the real world—is avoided. AR is rapidly emerging as a powerful tool for accelerating how front-line workers learn and conducting their work at substantially higher productivity levels.
AR-Enabled Engine Quality Control Check A worker is also guided to where the next procedure in the quality control sequence should best take place relative to the engine and where to hold the tablet, while their attention is drawn to use a tool to confirm the proper torque has been applied to these critical fasteners.
Capturing and Optimizing Human Work
In addition to enabling collaboration, training, and mentoring across distance and time, , AR technology solves two other important problems many companies face with front- line workers. First, how to capture the deep expertise of retiring workers as they depart, or to take advantage of experienced workers who are unable to travel. Second, how to enable all workers to perform a given process in the same way as the best, most experienced worker. And, as best practice evolves, AR can digitize process improvement and disseminate it to all workers to bring the whole team of workers along.
Central to AR’s value proposition, then, is its ability to capture, digitize, and store human expertise and work processes, which can be delivered later either for training as described above, optimized, or at the time and location of need to a worker operating in the front-line work environment.
AR wearable headsets are not only tools to deliver digital AR content to the physical work environment, but also powerful tools to efficiently capture and digitize human work processes and expertise from the physical environment. Experienced technicians and engineers use a wearable device to capture a task or perform the SOP as they carry it out. This replaces Frederick Taylor’s stopwatches in measuring physical work. And the content is then turned into a step-by-step guide with instructions for other workers to follow, through the same type of wearable device.
Using AR to capture and replay work processes is a huge conceptual leap beyond YouTube how-to videos. The digital knowledge of retired experts can be used not just to capture their own work, but to digitally shadow and coach new hires during their training.
Comprehension of expert content by the recipient is dramatically greater when AR maps the interactive digital content directly onto the actual work environment.
AR Playback of a Previously Captured Work Process A new worker watches visual content captured from an expert showing what operations she should perform at this machine, with relevant tips anchored in the physical environment and each step in the procedure.
A good example of work capture and exchange is Smiths Medical, a medical device manufacturer participating in the “Ventilator Challenge UK” to manufacture scarce ventilators in response to the COVID crisis. Smiths used AR to capture its front-line manufacturing process knowledge and procedures for ventilator production, and delivers those guided step-by-step instructions to production workers at GKN, an automotive and aerospace manufacturer. By capturing and distributing Smiths’s expert knowledge, the company is able to digitally guide workers from other companies and industries, with no medical device expertise, in making the same ventilators. There are many similar examples of companies which turned to AR during this emergency and were impressed by its utility.
The Emerging Platform for Front-Line Work
These uses of AR are major productivity drivers, in good times and bad. According to the Manufacturing Leadership Council’s 2020 Factories of the Future survey, gathered before the COVID-19 crisis, 48% of manufacturing companies had already adopted or planned to use AR in the next two years. During the crisis, thousands of additional companies utilized AR as a critical technology to enable their ongoing operations irrespective of travel restrictions and social distancing. Capturing and digitizing the expertise of experienced workers also establishes the foundation for optimizing the physical work processes of front-line workers. Case studies suggest that AR can dramatically reduce worker training time by 50%, increase worker productivity by 30-50%, and reduce human errors by 60-90%.
The coronavirus crisis has brought to light how technology can enable a distributed and mobile knowledge workforce to actually improve productivity as the nature of their work changes. AR provides front-line worker solutions that parallel the well-established digital technologies that have come to the rescue for knowledge workers. In a world where a remote workforce is the new normal, AR is a technology whose time has come. M
- Prioritization: AR is a broadly applicable technology. Dozens of opportunities exist across the value chain, from engineering, to marketing and sales, manufacturing, and field service, which requires disciplined use case prioritization to maximize business value capture. Start by defining measurable challenges that have financial consequences for your organization, documenting the as-is state to quantify the opportunity and later measure ROI, and then prioritize the AR use cases that best achieve that business value.
- Hardware: Different use cases call for different devices, so each AR solution should be designed with a device type in mind to ensure a good user experience. Start by deciding whether head-mounted or hand-held is best suited for your use case. While a standard mobile phone or tablet work well for many use cases today, hands-free operation will require head-mounted devices from Microsoft, RealWear, Vuzix, or others. While that technology is improving quickly, companies should develop an AR strategy that is compatible with many devices, to avoid getting locked-in.
- Content: AR solutions are only as good as the quality of the content and data that companies use. Getting access to the data or creating new content, and combining it in a meaningful way, can be challenging. Leading companies today reuse 3D content from CAD systems and leverage data from IoT systems to accelerate this process. Companies would also benefit from a digital thread— a single source of data truth by real-time synchronization of systems —to link interrelated data across all the processes, products, and people in the value chain.
1 Cognitive distance is the gap between the form in which information is presented, and the context in which it is applied. Greater cognitive distance increases the “cognitive load,” or mental capacity, required to interpret the information, hold it in the working memory, and translate it into implications in the physical environment.
