Virtual reality (VR) today is stuck in a box, a 4K 360 video frame box to be exact. Today VR is built upon a 4K 360 video-frame-based technology that is optimal for 360 video, but is definitely not optimal for gaming. Today’s VR gaming has to render about eight times more video than is visible in the headset, assuming a typical headset with 90-degree field-of-view. The rest of the rendered video represents the other scene around you that you don’t see until you turn your head. Because of the large amount of non-visible video rendering, VR computers today are specialized and need higher performance and cost more than typical computer systems.
If the VR is 3D, you only see about 1/16th of a 4K frame in each eye, or barely more than standard definition pixel detail. Compare that to the video gamer’s common favorite monitor resolution of QHD which is twice as much pixel detail as Full HD, or 8X more pixel detail than visible in today’s VR. Even if you use a high-resolution headset, the source video is still limited by the 1/16th of a 4K frame visible in each eye piece.
One reason why conventional video game companies wouldn’t want to jump into the VR market is that they have a certain level of detail and a certain framerate that they want to achieve to give their gamers the experience that succeeds in their customer base. To give their customers much lower resolution and much slower framerate resulting from converting to VR would make for unhappy customers.
To make conventional VR achieve today’s gamer’s preferred visual detail resolution (QHD each eye, or 4K passive 3D equivalent) would require 360 video frames with twice as many pixels as 8K video or half as many pixels as 16K video. A consumer VR computer to handle that rendering size and desirable high framerate would not likely be seen in the next decade. That means that movie Sci-Fi such as „Ready Player One“ with VR realism would not be realized for more than another decade.
Stepping out of the box, we would note that computer performance has increased dramatically since VR was first realized. The lower performance during VR’s conception made it a necessity to render a full 360 frame so there was no motion lag when turning your head. But, we are now seeing that a host of conventional video games can be rendered fast enough on an average consumer gaming computer to not have a perceived motion lag. This makes it possible to take a different approach to VR where only the view in the VR headset is rendered. As the gamer’s view is changed, the inherent game framerate is fast enough to keep up. As pointed out before, this reduces the equivalent amount of VR video rendering by a factor of 16. The one missing piece to augment the solution is a new A.I. Neural Net based technology which is capable of taking the gamer’s preferred full QHD resolution real-time gaming video stream and converting it into high-quality 3D with QHD resolution in each eye piece. This is accomplished with custom hardware realized in an FPGA chip that makes about 2000 computations per clock cycle which is a couple orders of magnitude greater than possible with a typical processor running software. This computation rate is required to achieve very-good easy-on-the-eye 3D quality, and it’s a principal reason that industry-wide real-time conversion to 3D has always previously been far lower in quality, and realistically, awful. The neural-net approach can create 3D that previously required tedious manpower intensive human artistic transformation, and it does that with less than 20ms latency which is undetectable by a human brain. It estimates the depth of every pixel in many of the same ways that a person can view a 2D image and estimate what is in front versus behind and the distance between. Implementing this in hardware eliminates any additional load from the computer or console hardware other than running the video game.
This new approach can be applied to much of the last 30 years of video gaming such as with Fortnite, Overwatch, Call of Duty, PUBG, Ghost Recon, iRacing, Halo and many more conventional video games to produce a 360 3D VR experience that has 8X more resolution visual detail than is possible with today’s conventional VR.
This new approach is really the only practical technology approach to achieve the industry and Sci-Fi vision for what VR could be where the difference between reality and virtual reality can become blurred. This approach is being demonstrated in a working technology demonstration system.
Obviously, this is a simplistic overview of how to achieve this new reengineered VR approach for easy initial reader understanding. And we expect this approach to be controversial because many have invested a lot of money in the current conventional and limited approach to VR. VR 2.0 can be increasingly disruptive to conventional VR in the future.
But this approach can be plug-&-play with any VR content or metaverse content that can be rendered to a single QHD monitor and supports head and body motion. Some will say that conventional video games are not designed for VR. But we’ve been able to compensate in custom hardware to create a great visual experience. Some will say that conventional video games don’t support 6 degrees-of-freedom (6 DOF). Realize that VR 2.0 can support any number of DOF that the game supports. And it will be easy for gaming companies to support more DOF in the future as this doesn’t effectively add more computer computation.
Today, the most popular VR configuration is seated VR which requires only a small amount of space in gaming centers and at home. Conventional video games using VR 2.0 approach can already provide this level of experience. And with VR treadmills such as Omni treadmill, and many others, can provide near 6 DOF motion with conventional video games effectively in an unlimited motion play space. Now we are not proposing that conventional video games are the only way to apply this new approach, but it is a way of experiencing it immediately along with most metaverse applications that can also output to a monitor.
Some people will ask what about headset barrel lens distortion, or other distortions? First in the hardware, it’s not hard to support a lens distortion compensation. But second, if you have ever looked at a 360-video frame, you will notice that it is peculiar. It is like that because it is cramming 360 degrees of visual space into a rectangular video frame. Essentially, the linear prerendered VR object space is rendered into non-linear spherical coordinate space mapped into the rectangular 4K frame. Then to view in the headset, a small portion of the 4K 360 frame is rendered back to linear space to view in the headset plus lens and other distortion compensation. This all adds additional computation load beyond just having to render all the view that is mostly not visible in the headset. And it also adds more distortion that needs to be removed as best as possible.
In contrast, the VR 2.0 model, renders linear game space as normally seen on a monitor and that is mapped directly to a linear 3D headset view, plus any needed lens compensation. In our V 2.0 model, we render to a linear view that can clearly see to the corners of the QHD headset displays which is using optics more like personal theater headset optics than a typical VR headset. This makes the visual experience crisp and sharp to the edges and corners where many conventional video games have resource indicators. Because the view is 16:9 aspect ratio, it feels more panoramic than the typical VR headset which has more of a square view. This makes it easy to play conventional video games in full 360 3D VR using VR 2.0. Plus, since the resolution in each eye is 8X greater than is possible with conventional VR, the gamer is looking at high-resolution 4K passive 3D equivalent experience instead of the more blurry looking standard definition level of pixel detail in today’s VR technology.
Also, note that while UltraVR can be played using conventional game controllers, carbine and driving controllers, and even keyboard and mouse, there is no reason why standard VR controllers can’t be used as well. Every new technology that is created to make VR better should in general be able to make VR 2.0 even better as well and will most likely be plug compatible.
You can see a gamer’s first impression using VR 2.0 technology at . VIZN Gaming (trademark) calls it UltraVR (trademark). While conventional video gaming is played on a monitor leaving the gamer a little unattached to the game action happening behind the monitor screen, but inside UltraVR, the gaming feels very intense as they are actually feeling like they are inside the video game with life-size opponents all around. The technology demo currently uses a laptop plus the custom hardware inside a 19in rackmount box, one of the product versions will resemble a small cube which includes a Windows gaming motherboard and the custom „Superconverter“ hardware and headset. Because the view is 4K passive 3D equivalent, there is no computer monitor required because a 1080p virtual monitor can be popped up inside the gamer’s virtual hangout/office space where they can run windows applications and launch games. Another model will work with existing gaming computers and consoles.
From a market perspective, enabling conventional video gamers to play their favorite video games in VR 2.0 technology is more than a 200X larger market than the VR market today. Just look at Fortnite with more than 350M players compared to a tiny fraction of players on the most popular VR game. And because VR games are relatively new, they typically don’t have the same sophistication of game play that exists in conventional video games that have been evolved and optimized for decades. Plus, since you don’t need the much more expensive VR computer, VR 2.0 fits into the volume gaming consumer market with many more potential buyers.
There is still much more that can be said in future articles about how all this works. I hope this quick overview gives you an idea of the prospective new opportunities that can be had with this new reengineered VR 2.0 approach. If you are interested, you can schedule and actually try out the UltraVR technology demo and experience popular video games in 360 3D VR.
Quelle:
Foto: Example system creates 360 3D VR of much of the last 30 years of video games with 8X more visual detail than VR
https://www.linkedin.com/pulse/virtual-reality-20-vr-reengineered-craig-peterson/
