If you’ve watched YouTube streams of conventional 2D videos — such as commentaries on games, TV shows, movies, or trailers — you know that the streams are visually flat: video of the content, sometimes with a person’s face superimposed in the corner for added discussion. But Apple appears set to change that paradigm for augmented and virtual reality streaming. A just-published patent application reveals that it’s been working on compositing multiple streams in a way that could let AR and VR viewers watch streamed content from their choice of angles.
While the premise uses complex technology, the end result is fairly easy to understand. Instead of just recording video and audio from a single perspective, such as the composited 2D view of live video from an iPhone’s or iPad’s camera with augmented content such as Pokémon in the scene, Apple would record separate streams that capture both the experience from the recorder’s perspective and time-stamped additional data that could be used to change the experience for viewers.
One example would enable a viewer to experience the AR/VR content from a different angle than the streamer, using the same 3D models and embedded positional audio details to recreate the scene appropriately from a mixed reality headset’s perspective. Another implementation would separately record multiple viewing angles that viewers could easily switch between. Depending on the hardware’s capabilities, other examples might include bringing aspects of the original scene — such as temperature, illumination, and human participants — into or out of the stream.
Apple’s goal is to create easily stored AR/VR recordings that have the “rich” depth to be experienced in multiple ways, potentially appearing as a web plugin that passively plays composited content like a video but morphs into something interactive when you engage with it. Given the company’s history with interactive iAds, this could be a way to engage users with advertisements or the next evolution of Twitch-style video streams for AR games. As this is a patent application — published today but filed last August — there’s no actual timeline for rollout of the technology, but it could tie in with the company’s reported work on AR/VR headsets or be implemented earlier as a unique way to preview certain Apple Arcade titles with AR support.
Quelle:
Foto: A demonstration of HomeCourt, an AR recording app for basketball player development, at Apple’s September 2018 special event. Apple
Apple seeks patent for AR/VR video recording with multi-angle playback
