Harness machine perception and artificial intelligence with Meta’s developer platform
Augmented and virtual reality (AR/VR) has always been an important employer for developers. In a survey back in 2017, it was revealed that 31 percent of the global developer community were involved in virtual and augmented reality projects.
This number has steadily increased – particularly during the pandemic – and Facebook (now Meta) estimates that it will create 10,000 new jobs in the EU alone, spurred by its vision for the metaverse.
As part of its commitment to aiding developer efforts, Meta launched the new Presence Platform at Connect 2021. The Presence Platform aims to consolidate all the Dev tools necessary to build AR, MR, and VR worlds for Meta Quest.
In the future, designers and software developers can rely on the Presence Platform’s prebuilt tools and capabilities to save efforts in building AI and ML.
You could call it almost an Apple Developer Program for the metaverse – a concerted endeavour to bring together app experts and enthusiasts to grow Meta’s virtual reach.
Defining the Presence Platform
The Presence Platform can be defined as a set of machine perception and artificial intelligence capabilities as well as sample projects that will help accelerate MR and VR scene, app, object, and interaction development for the Meta Quest environment.
Importantly, Facebook has not retired its Oculus for Developers portal and Presence Platform will not be launched in its entirety until well into 2022.
For now, it is the first step towards bringing together a community of independent and third-party developers to gradually expand Meta’s VR capabilities.
What Are the Key Features of the Presence Platform?
The Presence Platform was formally announced in October 2021 during Connect, but some of its features have been around for a while. For instance, the Passthrough API was first launched in 2020 and will now reach general availability via Presence Platform.
Here is a full list of its key features and capabilities:
1. Access to Passthrough technology
Passthrough is a mixed reality API that allows developers to blend virtual objects with real-world surroundings. At its core, it is a foundational technology that superimposes black-and-white images generated by VR headsets onto the real world.
The Presence Platform Passthrough API will enable custom styling, custom tints, and more controls for developers.
2. Guided by Responsible Innovation Principles
All activity enabled by the Presence Platform will be subject to Meta’s Responsible Innovation Principles. These were among the underlying philosophies behind Facebook’s new brand as announced at Connect 2021. Developers using Presence Platform will be urged to:
- Never surprise the user
- Provide meaningful controls
- Consider every user group without discrimination
- Put people first, particularly in terms of data rights
All of Presence Platform’s tools are built on these principles.
3. Awaiting Spatial Anchors Experimental
In a recent blog, Meta has announced that Spatial Anchors will be “available soon” in an experimental mode. This will allow developers to build and pin persistent digital objects onto the user’s real-world surroundings.
For example, if a user is working on an MR whiteboard, the whiteboard will remain exactly where it was the next time the user logs in.
4. Leveraging a Scene Understanding capability
Scene Understanding makes it easier to link virtual experiences with the physical environment. It will automatically detect if there is a wall in front of the user and will mount the virtual whiteboard accordingly.
Using Scene Understanding, developers can create scene-aware experiences on Quest. Importantly, this also includes Scene Model and system-guided Scene Capture flow to help capture physical surroundings with indexable and quarriable accuracy.
5. Using the Interaction, Keyboard, and Voice SDKs
Scene understanding is part of the Insight SDK component of the Presence Platform. In addition to this, there are three more software development kits – Interaction SDK, Tracked Keyboard SDK, and Voice SDK.
Interaction SDK empowers developers to create advanced hand controller-centric interactions such as grab, poke, target and select. Tracked Keyboard is meant to bring a typeable virtual keyboard into VR/MR spaces, while Voice SDK enables hands-free navigation.
6. Prototyping based on detailed documentation
One of the major features that developers can expect is detailed documentation for all tools, capabilities, and sample projects.
Interestingly, Meta has built a sample experience called The World Beyond, which it intends to release early next year. Developers can rely on this sample almost as a template to build their own custom experiences.
Oculus Platform vs Presence Platform
A question that might arise with the launch of Presence Platform is that, doesn’t Facebook already have an Oculus Platform?
Indeed, Oculus Platform has long served the developer community as its one-stop portal to design, build, and publish VR apps and is now in version 34.
The main difference is that the Oculus Platform SDK is meant for social VR use cases. It has APIs for quick invites, notifications, avatar management, and other core capabilities.
The Presence Platform, on the other hand, is purpose-built for mixed reality applications – particularly for scenarios that intersect virtual worlds and objects with real ones.
Meta has announced that the two Platforms are “designed to work together” and developers can use them together to:
- Build mixed reality user experiences for Quest
- Drive more natural user interactions on Oculus Quest devices
Note: While Oculus Quest is all set to be rebranded as Meta Quest, the company plans to stay with the current Oculus branding for the Oculus Platform for developers.
A Gateway to the Metaverse?
The Presence Platform marks Facebook’s first serious foray into mixed reality.
Beyond rudimentary transitions from the user’s physical surroundings into VR apps on Quest, we can now expect complex mixed reality scenarios, multi-user interactions in MR, and a host of other apps – even third-party ones.
Presence Platform projects could serve as the foundation for some of Meta’s other initiatives, including its futuristic AR/MR glasses, Nazare.
In other words, Presence Platform invites developers and VR professionals from around the world to contribute to Meta’s already impressive roster of VR capabilities. And, the underlying principles of democratisation and crowd-sourcing is exactly what’s expected to power the metaverse of the future.
Quelle:
