Microsoft holographic display technology HoloLens 2 and cloud-powered Azure Mixed Reality Services are the pioneering digital tools that power the mixed reality experience at space technology company MDA’s DREAMR Lab.
When retired astronaut Tim Kopra first stepped into the world of mixed reality at the MDA DREAMR lab in Brampton, Ont. it almost took his breath away.
It gave me the sense that I was back on a spacewalk
Looking at pictures doesn’t prepare you for the scale of a spacewalk and the International Space Station (ISS) — which is about the length of a football field, he says. Kopra should know; he spent 244 days in space during two separate missions in 2009 and 2015–16 and performed three spacewalks during that time.
MDA, an international space mission partner and space technology company, developed the Dynamic Robotic Emulation and Mixed Reality (DREAMR) Lab with mixed reality technology that gives users that “wow factor.” The lab has two robot arms that simulate how objects react in zero gravity and combines them with mixed reality visuals of space hardware, like satellites. And with 3D virtual models of the space station and its robotic arms, the users also get a sense of scale and spatial understanding that helps them problem-solve quickly.
The DREAMR Lab team is using these models to train astronauts, flight controllers, and engineers to operate the robots on the space station for accomplishing complex tasks in zero gravity.
To enhance this experience, users can put on a Microsoft HoloLens 2 headset and see a mixed reality 3D model of the International Space Station. According to Kopra, the experience makes you feel like you’re really on the ISS with the Canadarm2.
“Training for a mission with HoloLens 2 gives people the insights to understand these very complex mechanisms. Not just how they operate, but how they move and react in space,” says Kopra, now Vice President of Robotics and Space Operations at MDA.
The mixed reality experience helps people understand how to control the robots to perform tasks like moving cargo vehicles, doing maintenance on the space station, and capturing and servicing satellites.
The realistic training also helps avoid mistakes on a mission. “If things were to go awry, then the consequences can be very dire,” explains Kopra. “We have to make sure that we don’t damage space station structure, for example. Or when we capture these hovering vehicles outside the space station that we successfully latch onto them.”Microsoft HoloLens 2 and cloud-powered Azure Mixed Reality Services are the pioneering technologies that power the mixed reality experience at MDA’s DREAMR Lab.
The mixed reality holographic display unit uses cloud-based 3D bookmarks called Azure Spatial Anchors to allow people to position holograms they’re viewing relative to real objects.
“When you’ve got different people in different rooms or different physical sites, the Azure Spatial Anchors are providing a frame of reference so people can collaborate along the same point,” explains Sean Graglia, Azure Mixed Reality Lead for Microsoft Canada.
Mixed reality also helps astronauts see inside the robots and other parts of the ISS they’re working on. That’s because HoloLens 2 allows users to virtually strip away the outside of large, complex pieces of machinery in the hologram to see the positioning and movement of the mechanisms inside.
HoloLens 2 and Azure are powerful tools well beyond this immersive astronaut training. They’re design tools, they enable data processing and visualization, and they enable real-time team collaboration in 3D.
They’re equally useful on Earth, with applications in many different areas, including healthcare, manufacturing, and education.
In Montreal, healthcare specialists are using this technology to provide remote consultation for people with Covid-19. In the U.S., medical students are using them to visualize human anatomy. In Japan, whiskey manufacturers are using them to train workers.
At the DREAMR Lab, engineers are also using it to visualize data in a meaningful way. They can feed real-time data from a robot arm into its 3D hologram so that viewers can view it in real-time, allowing them to see what’s happening and troubleshoot.
“To be able to have a visualization tool instead of looking at a bunch of numbers or static models in the process of trying to resolve a problem, to be able to see it in this three-dimensional context is really important,” says Kopra. “We all tend to be very visual learners.”
MDA is also using HoloLens 2 as a 3D design tool. “We see it as a virtual rapid prototyping capability, and we’ve used it to identify areas in which a design may have some shortcomings,” says Kopra. The virtual model allows the team to slice away layers of the machine and see where the mechanism might have gotten stuck, for example.
“It’s a really important engineering tool to identify and optimize solutions as we go through the building of different elements of the Canadarm3,” says Kopra. MDA’s Canadarm3 will be a key part of NASA’s planned Lunar Gateway space station around the Moon. Preparations are ramping up for the initial launch, set for 2024.
“Especially in the last several months, we’ve been able to see the benefit and excitement … from training astronauts and flight controllers,” says Kopra of the mixed reality technology.
“It’s really starting to take off.”This story was created by Content Works, Postmedia’s commercial content studio, on behalf of Microsoft Canada.
Quelle:
Foto: The DREAMR Lab team uses 3D models created with Microsoft HoloLens 2 to train astronauts, flight controllers, and engineers to operate the robots on the space station for accomplishing complex tasks in zero gravity. PHOTO BY SUPPLIED