Microsoft Guides, the next step in training & guidance with Mixed Reality

It is more than 4 years ago that Microsoft came with the first release of Microsoft HoloLens device in the world of Mixed Reality. It took some time before Microsoft released several out-of-the-box applications for businesses. And they are ruling the Mixed Reality landscape since then. Their focus stays continuously on providing solutions which addresses the most common challenges for businesses.

Business challenges

One of these business challenges is training personnel. Training personnel in organizations require a lot of effort from experts on the workforce. And with a large staff turnover, something which happens a lot on the factory floor, is training a costly part of their business. But even keeping your workforce up-to-date with the latest requirements for training, or to prep students on school for what is coming or getting more insights by combining it with for example a Digital Twin will improve, support and saving costs for your organization allowing you to drive better business.

It is important for organizations to understand the business value of Microsoft Guides. Eventually organizations want to empower their workforce, optimize digital operations and deliver new services internally or externally to their customers. And that will result in a different way of thinking within manufacturing, training, support and guidance of employees.

Microsoft Guides

Microsoft Guides is a Mixed Reality business application which allows you to create a tailor made guide for employees on the factory floor. It allows you to create holographic work instructions which support a whole or part of the work flow. There are absolutely no coding skills requires for creating these Mixed Reality guides.

Microsoft Guides is part of the Microsoft Dynamics 365 family and therefor actually called Microsoft Dynamics 365 Guides. It uses the base storage system of Dynamics 365, its underlying Common Data Service and Power Apps to store its flow and content.

The solution works for Microsoft HoloLens 1 and 2. But understandable it is extended based on the new functionality provided by Microsoft HoloLens 2 like natural gestures and other. The natural gestures are currently only used for the author when building guide.

Since then the Microsoft Team responsible for Microsoft Guides is extending the functionality of the application and its platform on a monthly basis. They are keen on getting feedback from customers, MVP’s and consultants. And they take the feedback seriously.

How does it work?

In short a guide is created by an author. The author starts outlining a guide into different tasks using a desktop application. Outlining is based on requirements of the customer and the guidelines of building a guide. Then the author switches to the Microsoft HoloLens 2 device. There the guide is attached to for example an object in the real physical world by editing each task of the outlines. This is accomplished by moving, placing, sizing and rotating the holograms around in the real physical world. Adding additional holographic instructions like arrows, hands and more.

An operator logs in Microsoft Guides and select the guide. The operator needs to synchronize his training using a tag or positioning a holographic object on the object in the real world. After this the guide is executed through the outlines and tasks. The operator experiences the cards with the steps explained floating around. A tether (dotted line) is used to indicate where the step needs to be executed on the real assets. Finally the author is able to monitor and analyse the progress and execution. It allows the author to improve the guides and support the operator in improving their skills.

Different approaches

Microsoft Guides allows you to use different approaches.

• Real-life assets – This is the best way of having hands-on training for employees. The guide is synchronized on a real-life object using a tag or holographic object.
• Virtual assets – guides with virtual assets allow you to have education or training without the actual real-life asset. The asset is digitally placed as a holographic object in each task. This is specifically interesting in situations where you are not able to have the actual asset. It is also a more passive way of training since you are not able to execute the tasks on the real-life asset.

Before you start

One of the biggest mistakes is thinking that creating a guide is simply creating a few outlines and task. It is much more comprehensive. Before you start you will need to gather as much content as possible to create your guide. This is accomplished by having for example workshops and inspiration sessions with your customer or target group. There are a few things which are important for creating your guide. You need to understand the space where the guide is build for. You need to understand the procedure and workflow of the actual work. These things will influence in how you build your guide. If possible try to involve an expert of the organization. Actually it is a must to create a good guide. Keep that in mind!

Also try to understand the objectives and sub-objectives of your guide. This will influence the amount of outlines and tasks you want to use. You would think why? But it is important to not lose your operator using the training into doing too much tasks within the same outline. Something we will explain further in detail. And the final one is taking into consideration for which role you are creating a training. The role determines what you need to highlight from the asset and the depth of information you want to provide.

Anchor

Guides makes use of an anchor. The anchor is used to position the guide at the right location in the real world. This is accomplished by using a QR code or a Circular code. The later one is a predefined tag by the Microsoft Guides team. It is also possible to use a holographic object as a marker. But that requires to position the holographic object at the exact position which is in most cases very difficult. This is mostly interesting when it is not possible to place a tag on the actual asset.

Structure of a guide

The structure of guides exists of one or more cards placed in an outline. Your guide can have more than one outline. the outlines and cards are executed in a logical sequence. There are rules for creating fabulous and great working guides. Information about making guides great can be found here. Not going too much in detail, but it explains from begin to end and has a large number of great tips and tricks. Examples are that each outline must be a specific task within the work flow. It should have a clear beginning and end. It explains how to get around with holographic objects and to be consistent in using models, styles, texts and more.

The outlines and tasks are created via the desktop application. The desktop application allows you to add content. Content can be anything like images, sounds, video and holographic models. It offers out of the box several holographic helpful parts like arrows, generic tools, hands, numbers, symbols and animated zones.

Actions

A really cool feature which is just added two months ago is having actions in a card. At the moment there are only two actions. But i would expect more in the future. And who knows… maybe even add your own actions. But for now we have the action website link an Power App.

The website link allows you to add one specific link per card which can be opened during the presentation of the card. Imagine using a link to more information about what the employee is doing. Or maybe real-time information coming from a sensory device shown at the real object or in the virtual model.

Power Apps is a major cool feature. It allows you to show a Power App during a card in the outline. Just like the website link you can have a different Power App per card. Imagine a Power App with questions to be answered by the employee. You could for example use a single Power App with several questions. Each question can be reached by using a parameter in the URL to the Power App. And the results are stored in a database based on the credentials of the logged on user. There are so many more things you could do with this. And just like Microsoft Guides, Power Apps has a creator interface for power users. No additional coding and technical skills are required.

Roles and rights

At the moment Microsoft Guides knows two roles. We have the author and the operator.

The author is allowed to create, rename and edit guides. The author can also activate or deactivate guides. Just like the operator the author can also operate guides.

The operator is allowed to view and operate assigned guides via the Microsoft HoloLens.

Improve efficiency by analytics

The performance of the operator is measured due to data collection during the operation of a guide. Each gaze and commit interaction on buttons is measured. Time related information about the run, guide task and step are stored. This allows the author to view usage statistics and detailed time-track information via Power BI reports. These metrics can be used by the author to optimize the created guide. But it can also be used to see how the operator’s performance is improving doing the work. Sharing the result using Power BI Reports requires an appropriate license.

There are two default reports available. The process time-tracking report and the Guides usage report. These reports can answer several questions like

• Is daily guide usage changing?
• What is the most frequently used guide?
• What is the average run time per guide?
• How long is a guide run in minutes?
• How long did each task or step take?

Power Apps & Power automate

Microsoft Guides can be integrated into existing process and workflows using Power Apps and Flow. You can start a workflow or even use the “Create work record” event. This integration is mainly on the back-end of Guides and differs therefor on the Power App action discussed earlier.

Dynamics 365 Field Services

There is also an integration with Dynamics 365 Field services. It allows you to attach guides to Field Services tasks. This gives you the ability to complete work orders via Microsoft Guides.

Conclusion

Microsoft Guides is the tool for building great guides in Mixed Reality. While the tool is just released last year, it offers an extensive amount of functionality which is extended every month. Dynamics 365 and Power Apps are the base for Microsoft Guides. And do not get mistaken. Power Apps is going to be big and very important through the Microsoft landscape. The same is for Microsoft Guides as tool to build guides and more.