Spatial computing is an evolving form of computing that blends our physical world and virtual experiences using a wide range of technologies, thus enabling humans to interact and communicate in new ways with each other and with machines, as well as giving machines the capabilities to navigate and understand our physical environment in new ways. Apple’s entrance into the market is a big step for the technology, but the idea has been in development for years. Now, spatial computing is already starting to make an impact, including on communication and co-presence, manufacturing, gaming, human resources, media, sports and entertainment, and data visualization.
Earlier this year, Apple unveiled its entry into a completely new market segment. During the company’s June developers conference, Tim Cook announced the launch of the Apple Vision Pro, a head-mounted device with just as much processing power as a MacBook Pro and the company’s first spatial computer. During the keynote, Cook signaled that the device is part of Apple’s long-term vision and a shift from personal computing (the Mac) to mobile computing (the iPhone) and now into spatial computing (the Apple Vision Pro).
The announcement sparked both excitement and skepticism about whether betting on this new technology was prescient or outrageous. People wondered just what this technology was, what its current and future business applications might be, and what it might look like for it to become integrated into our daily lives and work.
Apple’s entrance into the market is a big step for the technology, but the idea has been in development for years. Here’s what leaders need to know.
A Working Definition for a New Way to Compute
Spatial computing is a term that many in the business world might have heard for the first time during Apple’s announcement, but it is by no means a new term. In 2003, researcher Simon Greenwold, then at the MIT Media Lab, defined spatial computing as: “Human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces.” Companies such as Amazon, Apple, Magic Leap (my former employer), Meta, Microsoft, and others have come up with their own definitions, but they basically boil down to this: Spatial computing is an evolving form of computing that blends our physical world and virtual experiences using a wide range of technologies, thus enabling humans to interact and communicate in new ways with each other and with machines, as well as giving machines the capabilities to navigate and understand our physical environment in new ways.
From a business perspective, it will allow people to create new content, products, experiences, and services that have purpose in both physical and virtual environments, expanding computing into everything you can see, touch, and know.
Sci-fi as this may sound, it’s already in use. One could argue that our mobile phones are primitive spatial devices. Many professionals in augmented reality (AR), virtual reality (VR), extended reality (XR), and artificial intelligence (AI) have been working on spatial computing for years.
In practice, the coming generation of spatial computing tools is arriving in the form of wearable headsets that have cameras, scanners, microphones, and other sensors built into the device, and make use of artificial intelligence, augmented and virtual reality technology, and the internet of things to create a more immersive relationship with computing. Users interface through hand gestures and finger movements, gaze tracking, and voice. GPS, Bluetooth, and other sensors make creating digital content with physical context possible. And while much of the focus is currently on headsets, projects like Google’s Starline offer a glimpse of other possibilities.
The goal of all of this is to expand the canvas of our screens, allowing users to work, collaborate, and engage with data and applications in whole new ways. If it catches on, it would be an evolutionary technological shift away from static devices that must hang on our walls, sit on our desks, or rest in our hands to devices that start to fade into the background and allow us to go back to focusing on the physical space around us, albeit augmented. But what are the use cases that will make people and businesses actually want to use it?
Where Businesses Are Using Spatial Computing
Spatial computing is already starting to make an impact include communication and co-presence, manufacturing, gaming, human resources, media, sports and entertainment, as well as data visualization. Below is a list of some of the ways spatial computing is already being used in these sectors. This list is not exhaustive but helps demystify how spatial computing is already being used and how companies, nonprofits, and organizations are starting to think about spatial computing and its impact on their business and audiences.
Communication and co-presence
The way we communicate through flat screens is bound to evolve. Technology has thrust our social interactions forward, so a move toward more immersive ways of communicating and experiencing co-presence might be the next logical step.
A recent demo of Meta’s codec avatars on Lex Fridman’s podcast interview with Mark Zuckerberg in volumetric form made headlines recently, giving the tech and business world a sneak peek of what’s on the horizon. Apple also announced that the new iPhone 15 will record spatial photos and video, one of the features highlighted during the launch of the Vision Pro. A stronger sense of presence via spatial video and shared memories could become a reality with spatial computing.
Spatial computing could also solve many communication challenges for businesses by using headsets, volumetric video, or avatars to provide better co-presence at work for remote teams, clients, and other stakeholders. For example, in 2019 BNP Paribas rolled out holographic meetings across their global offices using Magic Leap headsets. Bank clients were able to put on a headset and be offered real-time investment advice in consultation with the bank’s real estate team in hologram form, while being able to manipulate and alter the common plans modeled in a 3D projection of new real estate ventures. The company used spatial computing and visualization to help clients around the world view real estate with a consultant in that area.
In a related vein, Google developed Project Starline, which uses computer vision, machine learning, spatial audio, and real-time compression to present a person as though they’re sitting across from you at a table. You can talk, make hand gestures, use body language, and make eye contact in life size and three dimensions — all without wearing anything on your face. As part of Project Starline’s early access program, Salesforce and T-Mobile are using new prototypes that are gathering real-world data on how Project Starline can help distributed workforces stay connected. While the technology seems like magic, it’s really the collaboration and ideas that take shape in a shared mixed reality that is truly magical.
Manufacturing
Manufacturing was one of the first industries to embrace spatial computing. Several already use spatial devices to remotely assist workers, display virtual work instructions, and capture soft knowledge before operators retire. From assembling parts on the line to quality assurance and assisting operators, spatial computing has measured success.
Lockheed Martin, an American aerospace company, achieved a 93% reduction in costs on one part of the manufacturing process for the Orion space vehicle when they implemented augmented reality aids. What once took eight shifts of eight hours each was reduced to six hours via Microsoft HoloLens and Scope AR’s Worklink.
Gaming
Data from Newzoo estimates that there are approximately 2.7 billion gamers around the world — meaning that one in every three people on the planet identifies as a gamer. The gaming industry is worth more than the movie and music industry combined. A progression from gaming on consoles, tablets, or mobiles is a natural progression for spatial computing. Pokémon Go, an AR mobile game, has millions of daily gamers across the globe, and who would probably make the shift to a new kind of device if that meant playing their favorite game in a more immersive way without having a small mobile screen get in between them and the Pokémon they wish to catch.
Rec Room has already announced support for the Apple Vision Pro upon release, and Resolution Games made a recent announcement that they will support the Apple Vision Pro as well with one of their games called Demeo as one of the first spatial computing games that will debut when the Vision Pro launches next year. Gaming just seems like a natural fit for this progression in the way we engage with computing.
Human resources
Recruiting and HR is another area that has been using spatial computing. The Dan Marino Foundation is an example of a nonprofit that has used spatial computing to help neurodivergent students prepare for an interview. Instead of using poorly produced videos and outdated slide decks, they turned to spatial computing to help neurodivergent job candidates prepare for interviews.
The scenarios the candidates experienced ranged from friendly to hostile interviewers, and during the process, educators were able to track candidates’ responses, eye contact, body language, and engagement during the interview training. Plus, the students were able to practice in the physical world seeing a virtual interviewer in front of them. It allowed them to practice the interview process, work on eye contact, and deliver their answers so that when they got to the job interview with a human interviewer, some of their social anxiety might be reduced and they would be better prepared. Close to 72% of the foundation’s neurodivergent students who tried this training found employment. Compare that to national averages that indicate that more than 80% of adults with autism are unemployed.
Media, sports, and entertainment
AT&T knows that spatial computing will require new networking speeds like 6G. The company’s CTO, Andre Fuetsch, described new technologies like “volumetric spectrum sharing and efficiency” are needed to make 6G a reality. Magic Leap partnered with AT&T in 2020 to create in-store experiences for Game of Thrones’ The Dead Must Die: A Magic Leap Encounter. The physical space looked like King’s Landing, but when visitors put on the Mag Leap One, it turned into an icy scene where they would have to fight a virtual White Walker in the physical world.
In 2023, Apple and Disney announced a Disney+ app for Apple’s Vision Pro. Additionally, when Apple acquired NextVR in 2020, they signaled a focus on immersive sports experiences in the future. The way that we experience movies, TV, sports, and entertainment in general is impacted by our mobile phones. Now, think about how spatial computing will once again change our content viewing habits.
Data visualization
Data visualization is a clear use case for spatial computing. Even though designers create 3D prototypes with 3D design tools, they still have to show them on 2D screens instead of trying to read graphs or update dashboards to view data in context to the physical environment. BadVR does this with their product SeeSignal, which uses spatial computing to visualize cellular dead zones. Instead of carrying multiple pieces of equipment and reading a heat map, operators can use a headset to see in real time and real space where dead zones are.
Finding the Business Value of Spatial Computing
It’s easy to see why the business world has yet to start taking spatial computing seriously when generative AI has taken over the headlines and VC investments. Still, spatial computing offers some initial glimpses into how it will be useful today and what the potential business will be in the next decade. It is important to point out that current XR headsets already have AI, but the confluence of all advancements in technology makes the use of improved spatial computing technology using new breakthroughs in AI a revolutionary happening.
A recent Tweet (X Post) from Andrew Schwartz, Director of Metaverse Engineering at Nike, points out why spatial computing could be transformative. “If the organizing principle of the internet is that information wants to be shared, and the organizing principle of the metaverse is that information wants to be experienced, spatial computing brings together the tools necessary to create those experiences,” he wrote.
Spatial computing is a natural evolution of the way we use and experience technology. It can create new interfaces and ways of interacting with people. Companies will need to take spatial computing seriously, maybe not today, but in the next decade, because it will change how people interact, especially as spatial computing starts to come into our lives, much in the way that phones came into our lives and changed the way we interacted with each other and with our work. It has the potential to fundamentally change how people think about computing, from collaboration between co-workers, to supporting creativity and optimizing business processes.
The world is continually transforming into a blended physical and virtual one. Younger generations are growing up used to digital assistants, online video games, and augmented social media. They’re the next generation of customers, employees, and owners. Investing in spatial computing not only assists with business’ pain points today but sets them up for the future.
Employees uplevel their work when technology adapts to make sense for the person, not the person needing to make sense of the technology. Spatial computing isn’t a virtual escape. It’s not a fad technology. It’s the natural next step for how computers and humans will interact. We are still in the early stages spatial computing. Still, all signals point to a transformation as powerful as the one we experienced with the coming of the internet and the advent of the mobile phone. We are in the very early days of how these new interfaces that will change business and society in the years to come.
Quelle:
Foto: HBR Staff/Damir Khabirov/ Andriy Onufriyenko/Getty Images
https://hbr.org/2023/11/what-leaders-need-to-know-about-spatial-computing
