Tag

Apple

Vision Pro: How much can Apple’s futuristic AR headset really do?

It took a long time, but finally Apple’s augmented reality (AR) headset, the Vision Pro, is available outside the US in parts of Europe and Asia.

Offering what the company calls spatial computing, the high-tech headset aims to bring digital content into real environments and establish a new computer platform. All for a hefty price tag of €4,000/$3,500. So what exactly can they do?

From the outside, the Vision Pro looks a bit like ski goggles with an aluminium frame. A display fills the front. The battery is stored externally, reducing the weight of the headset, which means you can wear it for longer without tiring.

To ensure a tight fit, the face is scanned in a similar way to setting up Face ID on a smartphone. This selects the right light seal, which then shields the field of vision from external light.

From a purely technical point of view, the Vision Pro could be called virtual reality (VR) headset, but the device also overlays digital objects in the real environment so that’s where’s the AR comes in.

Once you put them on, you can see the room you’re in along with app icons floating some distance away. The Vision Pro has several cameras that capture the environment, detect movements, and record the position of your arms, hands and fingers. The image is then transmitted to the screens in front of your eyes. Each of the two displays has more pixels than a 4K TV.

Go anywhere — without leaving the room

You can also be transported somewhere else entirely by turning the digital crown on the right side of the headset. Then the real environment gradually fades into the virtual image. If there are other people in the room, they are also faded into the artificial environment.

Pressing the crown also calls up the app icons. There’s a second button on the left side that you can use for clicking purposes. The Vision Pro has biometric identification that recognises people by the pattern of their iris. It can also save the iris settings of different people so that you don’t have to calibrate the device every time.

There is a speaker on each ear that plays 3D audio. The direction from which you hear the sound adapts precisely and without delay to the position of the source.

To operate the Vision Pro, you have to learn one gesture in particular. Pressing your thumb and index finger together has the effect of a mouse click.

Your gaze takes on the role of the cursor: Whatever you are looking at is what you click on. For example, you can write on a virtual keyboard where the buttons seem seem to give way slightly as you tap them.

One of the Vision Pro’s superpowers is to make huge displays float in space. This means you can watch videos in giant format with a cinematic experience. While YouTube and Netflix are currently only available using the web browser, the streaming service Disney+ has an app for the Vision Pro that you can use to watch 3D films.

Watch immersive videos — and make your own

The Apple TV app also offers a sample of immersive videos. There are elephants at a watering hole, parkour runners in Paris, and a woman on a zip-line between mountain cliffs. Above all, however, you can experience sports such as American football, soccer, and basketball from the centre of the action like never before.

If you have an iPhone 15 Pro you can make your own immersive videos to watch in the Vision Pro. This provides a pretty unique way to relive precious memories such as a child’s first birthday or an enjoyable holiday.

With conventional images and videos, the Vision Pro has the advantage of being able to display them in large format. And anyone who has collected panoramic images with their iPhone can unfold them in the Vision Pro to create all-round views in canvas size.

The Vision Pro’s ability to fill the room with displays also pays off when working. The windows for email, messages, and other apps can be arranged next to each other and made larger or smaller.

Apple’s Bluetooth keyboard and trackpad can be linked directly to the Vision Pro and used for writing and controlling. You can place them on a desk and see them thanks to the cameras built into the headset.

You can also connect a Mac computer to the Vision Pro, which then takes on the role of the computer display with much more screen space and high resolution. The MacBook’s keyboard and trackpad then work seamlessly for both the programs on the computer and those that run directly on the headset.

For office workers who want more screen space, this works extremely well. You can also work with 3D objects, for example in computer design applications.

3D video chat using your own avatar

There is also a 3D function for video conferencing. To do this, you hold the headset in front of your face and have to turn your head in different directions. The result is a kind of hologram that is then used when making a call.

Thanks to the cameras at the bottom of the Vision Pro, this personal avatar follows your own facial expressions in real time. 3D holograms of your conversation partners also appear.

When it comes to games, the Vision Pro relies heavily on Apple’s in-house subscription platform Arcade. Hardcore gamers would probably be better off with virtual reality on the PC such as Index from Valve.

For €3,999/$3,499 you get the version with 256 gigabytes of memory. With 512 GB, the price rises to €4,249/$3,699 and the top model with one terabyte of storage costs €4,499/$3,899.

The next version of the visionOS operating system, due in the autumn, will bring even even more features, including the ability to transform conventional images into spatial photos and a travel mode, which was previously only available for airplanes, expanded to support use on trains.

So what about the Vision Pro’s competitors? Facebook group Meta has been working on its Metaverse virtual world for some years. Their Quest 3 headset costs €550/$500 while HTC offers the Vive XR Elite model for €1,399/$1,100.

In comparison, Apple benefits from higher resolution displays and from the fact that the Vision Pro can work with other Apple devices such as MacBooks.

Those two factors mean that the device can do much more than previous VR headsets. The ability to use huge virtual screens for office applications and spatial videos for private memories are likely to be game changers.

Quelle:

Foto: This is the most important gesture when using the Vision Pro: Bringing your thumb and index finger together does what a mouse click does. Zacharie Scheurer/dpa

https://www.yahoo.com/tech/vision-pro-much-apples-futuristic-101148637.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAAB_AGOYg0aFVdfXIGLzAYSkbe3HNjyHfdhMqt8ewgowrgktRICrQLwhlWYlzV_QMHsp-NdpyFFsK-r8cByC1fXvUTG1e17ZUojd_ZMz1Z71illevmiM1w2mduqqSqplLRSN1YvLwxG1vf86U3lB6QZrF6GcFXoBUc70bgsV2TUNQ

3. September 2024 0 comment

Examples

Top 5 enterprise use cases for Apple/Meta headsets

Since Apple introduced its Vision Pro headset in June 2023, enterprises have been hesitant to adopt it. Analysts are still awaiting the emergence of killer apps that could drive widespread adoption of the VR headset. While businesses have identified potential use cases, they remain lukewarm about making significant investments until these compelling applications emerge.

Similarly, Meta Quest 3, which debuted in October 2023, has gained more traction among gamers. However, on the enterprise side, it shares a similar reception as Apple’s headset. It’s worth noting that the Quest 3 is more affordable, with the top-tier 512GB model priced at US$649.99, compared to the Apple Vision Pro’s US$3,499 price tag.

As it stands, the question of which device will offer enterprises the best value remains open. Here are five areas where businesses can potentially capitalise on these cutting-edge AR/VR headsets:

Training and simulation

Hands-on, real-world demonstrations are irreplaceable for training engineers, nurses, or technicians. However, global businesses now face the challenge of operating seamlessly across geographical barriers and overcoming operational hurdles.

This is one area where AR/VR headsets, like the Apple Vision Pro and Meta Quest 3, can be particularly useful — particularly in training and simulation. KLM Royal Dutch Airlines, for example, has adopted the Apple Vision Pro to improve its technical maintenance training.

KLM Royal Dutch Airlines’ Engine Shop app aims to improve technical maintenance with spatial computing. Image courtesy of Apple.

The Vision Pro enables technicians to engage in detailed simulations using the Engine Shop app, which provides step-by-step instructions overlaid on 3D models of engine designs. This approach enhances the training process, reduces reliance on physical equipment, and helps minimise errors.

“We see Apple Vision Pro as a tremendous value-add that will improve our fleet availability and operations,” remarked Bob Tulleken, KLM’s vice president of Operations Decision Support. “Training our employees with spatial computing will lead to fewer costly errors because the most current information they need to do their job is there in front of them as they perform the task. This means we not only get vastly more efficient in our work, but also provide a better work environment for our employees to succeed.”

Building information modelling

Design flaws in the architecture, engineering, and construction (AEC) industry can be costly, especially if identified after a building is completed. To address this, firms are increasingly using digital twins, which allow them to visualise designs more realistically and spot errors before construction begins.

One of the companies leveraging digital twin technology is Norwegian architecture firm A-lab, which has replaced renders and sketches with 3D models in VR. A-lab uses SentioVR, an independent software vendor, to create virtual walkthroughs for its proposed projects. To visualise the environment, prospective clients don Meta Quest 3 headsets.

The Meta Quest 3, an advanced AR/VR headset, seeks to enhance immersive experiences in education, gaming, and more. Image courtesy of Meta.

The tech partnership was especially helpful for one of A-lab’s clients, which needed buy-in from local politicians for a redevelopment site.

“Using VR, the project’s outcome is going to be much better. We can incorporate feedback and speed up the design process,” noted Angie Mendez, the company’s Technology Director.

Real estate

Finding a house or an office space used to be a labour-intensive task, involving multiple property visits, scheduling viewings with owners and real estate agents, and spending significant time and energy. The introduction of AR/VR technology has changed the game both for property sellers and buyers.

For pre-selling condos, VR simulations offer clients an immersive experience of what it is like to live inside the property space. With the use of a headset like Apple’s or Meta’s, people can enter every space inside the virtual render.

For finished properties, clients can overlay digital information on physical spaces using AR technology, helping them determine whether certain furniture can fit into specific spaces or if the rooms are large enough for their needs. They can also analyse property details, assess damages, and receive home improvement recommendations.

AR/VR headsets also provide convenience for developers and realtors, allowing prospective clients to access all their properties while staying in a single location.

Collaboration and productivity

With the rise of work-from-anywhere set-ups, businesses and individuals need the flexibility to be creative and productive without being in the same physical space. To support this, Apple has partnered with developers to integrate collaboration and productivity tools into the Vision Pro headset.

SAP Analytics Cloud, for one, allows employees and executives to access their data dashboards and organise critical workflows within their virtual space. Through the Vision Pro, users can also explore data with contextual 3D maps and graphics.

“We are utilising Apple Vision Pro’s spatial computing capabilities with SAP Analytics Cloud and SAP Mobile Start. We believe that combining visionOS with generative AI will significantly enhance how enterprises operate. With SAP Analytics Cloud running on Apple Vision Pro, our customers can gain deeper insights from complex through more intuitive processes, and SAP Mobile Start offers access to all of SAP’s business applications and workflows in new, more effective ways,” stated Philipp Herzig, chief AI officer, SAP SE.

Meanwhile, Microsoft 365 and its AI companion Copilot, optimised for use with Apple Vision Pro, is said to offer benefits such as better data visualisation in Excel and easier content switching in MS Teams.

“Spatial computing has prompted us to rethink how professionals use AI tools. With Microsoft 365 and Teams on Apple Vision Pro, users can view and manage apps side by side, allowing for better organisation and coordination,” said Nicole Herskowitz, Vice President, Microsoft 365 and Teams.

Additionally, Apple introduced new functionalities for WebEx, Zoom, and Box on the Vision Pro, focusing on enhanced spatial computing capabilities for better virtual collaboration and data management.

Education

Traditional teaching methods cannot be replaced, but they must be supplemented in a digital-first society. Students are using tools like ChatGPT to simplify repetitive tasks, and teachers are increasingly looking for more innovative ways to capture kids’ short attention spans.

For a generation raised on platforms like YouTube and TikTok, AR/VR presents a significant opportunity to bridge learning gaps and present educational materials in a new and engaging way.

In 2023, Meta, in collaboration with VR Heroes and Poland’s National Research Institute (NASK), launched a series of free workshops in Poland to introduce AR and VR teaching techniques to educators. These workshops were designed for teachers at all levels and were held at Meta’s Warsaw community hub. They featured Meta Quest 2 headsets and dedicated educational apps, aiming to show how these technologies can be integrated into the classroom.

Teachers in Warsaw explore new AR/VR teaching techniques using Meta Quest headsets during a workshop aimed at enhancing educational experiences. Image courtesy of Meta.

The training also included sessions led by NASK, which explored the impact of VR on students and provided best practices for the safe use of these technologies in educational settings. The goal was to equip teachers with the skills to leverage AR and VR in developing students’ creative skills and digital competencies.

According to academic research, VR can improve various learning outcomes such as comprehension, memorisation, student engagement, attention span, and motivation.

“Using AR and VR as educational tools changes the way of learning, increasing students’ cognitive processes and their engagement. I look forward to following the progress of the Meta VR and AR for education programme in Poland. I appreciate that Meta provides modern technological solutions and know-how to develop this project,” said Mark Brzezinski, US Ambassador to Poland.

In the broader context, the adoption of AR/VR headsets by academic institutions remains uncertain. Schools and universities will need to consider the return on investment and other priorities, particularly in regions where access to basic educational resources is still limited. The decision to integrate these technologies will depend on a careful assessment of costs and potential benefits.

Quelle:

Foto: The Apple Vision Pro, Apple’s latest AR/VR headset, features advanced optics for immersive spatial experiences. Image courtesy of Roméo A.

Top 5 enterprise use cases for Apple/Meta headsets

2. September 2024 0 comment

Examples

Surgeons perform UK’s first operation using Apple’s Vision Pro headset

Apple’s mixed reality device could revolutionize the industry

What just happened? Apple’s Vision Pro might have its detractors, not least due to the $3,500 price tag, but its use in the enterprise and other non-consumer sectors could be revolutionary. In the UK, for example, surgeons have performed the first operation in the country where the mixed reality headset was utilized.

Apple hopes that one area where the Vision Pro will find plenty of success is the medical industry. The profession is examining the use of mixed reality in certain fields and, according to some analysts, Apple’s powerful headset could be a game changer.

During a recent operation to repair a patient’s spine at the private Cromwell Hospital in London, a scrub nurse working alongside the surgeon used the Vision Pro to help prepare, keep track of the procedure, and choose the right tools, reports the Daily Mail. This marked the first operation in the UK where the Vision Pro was used.

The software running on Apple’s headset during the operation comes from US company eXeX, which has made similar programs for Microsoft’s HoloLens. It offers nurses and technicians both holographic and touch-free access to the surgical setup and the procedural guides from within the sterile field of the operating room, according to the press release. The software also tracks each stage of an operation and can measure how well the op went compared to previous procedures performed by other surgeons.

„It eliminates human error and eliminates the guesswork,“ said Suvi Verho, lead scrub nurse at London Independent Hospital. „It gives you confidence in surgery.“

While this marked the first time that the Vision Pro was used during a UK surgery, the first-ever time the device was used in an operating room was last month, just three days after its release, when Orlando resident and world-renowned Neurosurgeon Dr. Robert Masson wore it during several spine reconstruction surgeries (top of page).

„We are in a new era of surgery, and for the first time, our surgical teams have the brilliance of visual holographic guidance and maps, improving visuospatial and temporal orientation for each surgical team and for each surgery in all specialties“, said Masson.

The Vision Pro is being trialed across different areas by several healthcare organizations around the globe. In addition to aiding surgery, it’s also being used for radiology, patient monitoring, medical teaching and planning, therapy, and spotting early signs of conditions.

Quelle:

Foto: Serving tech enthusiasts for over 25 years.
TechSpot means tech analysis and advice you can trust.

https://www.techspot.com/news/102212-uk-surgeons-perform-first-operation-using-apple-vision.html

19. März 2024 0 comment

Examples

Leaked Video Shows Browsing Safari in Apple Vision Pro Is Super Smooth

This isn’t the same 2D Safari web browser you’re accustomed to.

The latest rumors suggest that Apple Vision Pro is launching by the end of February, so it’s no surprise that leaks are flooding in.

Thanks to videos from X user @M1Astra, we have a better look at how much more multitasking you can accomplish with Apple’s Safari web browser inside of visionOS.

https://967490cfc4c62bca62d86967cbc74519.safeframe.googlesyndication.com/safeframe/1-0-40/html/container.html

Instead of a static 2D web browsing experience, Safari on the Apple Vision Pro lets you plop multiple windows and tabs in 3D space inside of the headset. This means a fully customizable work experience, whether that means a simple two-window setup or five windows for extreme productivity.https://platform.twitter.com/embed/Tweet.html?creatorScreenName=byjacksonchen&dnt=false&embedId=twitter-widget-0&features=eyJ0ZndfdGltZWxpbmVfbGlzdCI6eyJidWNrZXQiOltdLCJ2ZXJzaW9uIjpudWxsfSwidGZ3X2ZvbGxvd2VyX2NvdW50X3N1bnNldCI6eyJidWNrZXQiOnRydWUsInZlcnNpb24iOm51bGx9LCJ0ZndfdHdlZXRfZWRpdF9iYWNrZW5kIjp7ImJ1Y2tldCI6Im9uIiwidmVyc2lvbiI6bnVsbH0sInRmd19yZWZzcmNfc2Vzc2lvbiI6eyJidWNrZXQiOiJvbiIsInZlcnNpb24iOm51bGx9LCJ0ZndfZm9zbnJfc29mdF9pbnRlcnZlbnRpb25zX2VuYWJsZWQiOnsiYnVja2V0Ijoib24iLCJ2ZXJzaW9uIjpudWxsfSwidGZ3X21peGVkX21lZGlhXzE1ODk3Ijp7ImJ1Y2tldCI6InRyZWF0bWVudCIsInZlcnNpb24iOm51bGx9LCJ0ZndfZXhwZXJpbWVudHNfY29va2llX2V4cGlyYXRpb24iOnsiYnVja2V0IjoxMjA5NjAwLCJ2ZXJzaW9uIjpudWxsfSwidGZ3X3Nob3dfYmlyZHdhdGNoX3Bpdm90c19lbmFibGVkIjp7ImJ1Y2tldCI6Im9uIiwidmVyc2lvbiI6bnVsbH0sInRmd19kdXBsaWNhdGVfc2NyaWJlc190b19zZXR0aW5ncyI6eyJidWNrZXQiOiJvbiIsInZlcnNpb24iOm51bGx9LCJ0ZndfdXNlX3Byb2ZpbGVfaW1hZ2Vfc2hhcGVfZW5hYmxlZCI6eyJidWNrZXQiOiJvbiIsInZlcnNpb24iOm51bGx9LCJ0ZndfdmlkZW9faGxzX2R5bmFtaWNfbWFuaWZlc3RzXzE1MDgyIjp7ImJ1Y2tldCI6InRydWVfYml0cmF0ZSIsInZlcnNpb24iOm51bGx9LCJ0ZndfbGVnYWN5X3RpbWVsaW5lX3N1bnNldCI6eyJidWNrZXQiOnRydWUsInZlcnNpb24iOm51bGx9LCJ0ZndfdHdlZXRfZWRpdF9mcm9udGVuZCI6eyJidWNrZXQiOiJvbiIsInZlcnNpb24iOm51bGx9fQ%3D%3D&frame=false&hideCard=false&hideThread=false&id=1740994932305973418&lang=en&origin=https%3A%2F%2Fwww.inverse.com%2Ftech%2Fapple-vision-pro-safari-web-browser-video&sessionId=9af8527bb075af5e0ff488372f653c0742a5f2be&siteScreenName=inversedotcom&theme=light&widgetsVersion=2615f7e52b7e0%3A1702314776716&width=550px

As Many Tabs As You Want?

You’d think a flood of tabs would get hairy to operate, but the demo shows off the smooth operation that lets you switch between open windows and then the individual tabs within them. You can also drag out tabs to make new windows and reposition and resize them as you please. The demo shows up to five windows at a time, but there doesn’t seem to be a limit to how many you can open. That’s perfect for those of us who chronically have dozens of tabs open at once.

On top of the freedom to set up your virtual workspace as you want, the demos show off how the Vision Pro can implement 3D objects into Safari. In @M1Astra’s video (taken down due to copyright claims but still viewable on Discord) you can view a 3D render of the Vision Pro in 360 degrees and even resize it. The 3D object even casts a shadow onto the other tabs to indicate that you have it selected. While minor, this feature helps you get a quick overview of an object’s scale or design.

Apple Vision Pro demo — Apple’s trailer for the Vision Pro gave us a sneak peek at how Safari looks in virtual reality, but not the whole picture.Apple

Safari For Spatial Computing Is Coming Soon

You’ll be able to try out Safari on Vision Pro soon enough, considering Apple’s “early 2024” release is any day now. Release date rumors are constantly changing, but the last rumors claim production is ramping up for a launch by the end of February. There’s one thing that’s not in question: the $3,500 price tag.

We’re eager to try Safari for spatial computing in person and see how it differentiates from the 2D Safari experience on Macs, iPads, and iPhones.

Reliving My Memories in Apple Vision Pro Almost Brought Me to Tears

Looking at spatial videos, panoramas, and regular photos and videos in Apple Vision Pro is going to be major selling point.

I am more and more convinced that Apple Vision Pro will be the breakthrough device Apple is promising when the $3,500 “spatial computer” launches early next year.

Yesterday, I had the rare opportunity to try Apple Vision Pro for the third time since its announcement at WWDC 2023. The situation was the same as the last time: a short viewing session to look at spatial video, panoramas, photos, and videos. The difference this time, however, was that I was allowed to bring my own content — including spatial video shot with an iPhone 15 Pro — and take a look at it using visionOS’s Photos app.

I Got Emotional

Since the release of iOS 17.2 developer beta 2 for iPhone last month, I have been stockpiling spatial videos taken with my iPhone 15 Pro. If you own an iPhone 15 Pro or 15 Pro Max and want to learn how to shoot your own spatial videos, I recommend reading that because you do need to turn the feature on within the camera section in the Settings app.

As I noted last time, recording spatial video with an iPhone 15 Pro is easy once you’ve turned it on. You simply tap the Apple Vision Pro icon that appears in the regular video mode and then hit the record button. Videos are saved as normal 2D videos in your camera roll, but view them in Vision Pro and they have depth like any 3D video.

Prior to my viewing session, I was encouraged by Apple to capture spatial videos that would have sentimental value to me — family, friends, pets. These are the types of things you’re more likely to appreciate in 3D than, say, the burger you had for lunch two weeks ago. While I did bring content that had a higher chance of making me choke up, I also brought a few spatial videos that were a little more mundane — for example, walking through a train station tunnel or a short clip of my colleagues at a pizza shop — just to see what that would look like.

The first thing after going through a short setup process (using an iPhone to scan my face like you would setting up Face ID and also eye and hand tracking within the Vision Pro) was AirDropping my content to the Vision Pro. I was allowed to bring 15 photos, videos, panoramas, and spatial videos in total. I AirDropped my content from my iPhone 15 Pro to an Apple representative’s iPhone, which was then AirDropped to the Vision Pro I wore. AirDrop on Vision Pro is a similar experience to existing Apple devices. An AirDrop icon appears at the top of the “screen” and you look at it and pinch your thumb and index finger together to “click” it. A glassy, translucent Control Center then appears with the option to accept the AirDropped content.

I’ll get to looking at the non-spatial video content in a second. It’s not that they didn’t elicit strong emotions or anything, just that they show up as plain 2D in Vision Pro. On the other hand, spatial videos hit different. Maybe it’s because they have subtle depth to them — I noted in my viewing session last time that the 3D has just the right amount of depth, not too strong or too weak — or because you can look at them in an “Immersive” view where the border of the video becomes glowy and dream-like to give it characteristics of a memory. Either way, spatial videos feel alive. The dream-like memory border sells that feeling pretty well.

I kept tilting my head a lot, almost in disbelief at how surreal it was to see my mom in spatial video.

In one spatial video, my mom and I were having dim sum at a restaurant, and I was explaining to her what the Apple Vision Pro is and what it does. It was recorded last weekend, so the memory was fresh in my mind. Rewatching the video inside of the Vision Pro, it was as if we were transported back to the restaurant, sitting across from each other over a table of dishes. I kept tilting my head a lot, almost in disbelief at how surreal it was to see my mom talking, laughing, and eating in spatial video. My mom was who got me interested in technology, and I don’t think I would have a career writing about new consumer tech if not for her interest in it. To me, these convos are very precious, so to see them replayed with a sense of presence really tugged at my heartstrings. At one point, I fought back a few tiny tears, if only because there were three Apple reps sitting next to me. Self-aware of EyeSight and the possibility that they might be able to see my tears, I asked if they could see my eyes on the Vision Pro’s outside display. I was told they couldn’t. Pre-release software, you know? I obviously couldn’t confirm that myself as the person wearing Vision Pro. At a certain distance and window size, spatial videos can look life-sized. But even when I “pushed” the video window farther away (enabled by looking at the bar at the bottom of the window and then pushing it farther from me), seeing my mom in 3D made me emotional. I even laid back on the sofa and placed the virtual video on the ceiling.

I think Apple was right in suggesting I capture sentimental content in spatial video. Cool as it was to see my colleagues at the office and a bar with a little bit of the third dimension, I don’t think rewatching those clips had the same impact on me as the ones of my family. I suspect once people have the opportunity to see their own content in Vision Pro, they’ll get a better sense of when and when not to tap the spatial video icon in the camera app on their iPhones.

Looking at panoramas in Vision Pro is its own thing. I brought a few specific panoramas to see if it’d feel like I was brought back to the locations. The first was a shot in Arthur Ashe Stadium during the U.S. Open in 2019. The match was between Serena Williams and Maria Sharapova and POV was from the mid-row seats. Shot with an iPhone XS with 32 megapixels of resolution, the Vision Pro did a little bit of de-warping to the panorama to curve it in front of me. It looked good and expansive, but not nearly as surreal as a spatial video. I could use two hands to pinch and zoom in and out of the photo.

However, even larger and higher-resolution panoramas, taken with newer iPhones, look even better. Two others — a 51-megapixel pano shot at the top of Bear Mountain in New York and a 38-megapixel pano shot at Apple Park last year — were wider and, as a result, wrapped more to the sides and almost behind me, giving me more horizontal pixels to turn my head and look at. If you haven’t been taking panoramas of epic sunsets, beaches, and landscapes, I highly recommend you start.

As for 2D photos and videos, they appear as they would on a computer, either in a smaller window or as a fullscreen experience. I looked at a few photos of our family Yorkie, whom we had to put down earlier in the year, and although I tried to hide it, I felt a few waves of sadness viewing photos of her on such a big virtual screen. Mostly because I miss that little pup every day, but still, the clarity of the virtual screen in Vision Pro is engrossing. Which brings me to…

https://967490cfc4c62bca62d86967cbc74519.safeframe.googlesyndication.com/safeframe/1-0-40/html/container.html

The Best Virtual Big Screen

I said it many times before, and I’ll probably repeat it until it’s released: Apple Vision Pro is the best virtual big screen I have ever used. Even though I’ve only used the Vision Pro three times, each in brief sessions lasting around 30 minutes, I’m certain it’s the best (at least in a consumer product). When I say the screen looks like a proper big screen projection floating in front of you (or on your ceiling if that’s what you prefer), I mean it. It’s not like other VR headsets or video glasses, where they claim a display that resembles one measuring a hundred-something inches, but in reality, it just looks like a much smaller TV screen or monitor. It makes even the virtual display inside the PlayStation VR 2 look budget.

Apple Vision Pro is the best virtual big screen I have ever used.

While I’ve spent most of my Vision Pro sessions looking at the Photos app, I have no doubt that the mixed reality big screen experience will translate really well for viewing other types of content like movies and games. Frankly, I think everyone is so focused on comparing Vision Pro to the Quest 3 that they’re missing the point of what the Vision Pro really is: it’s going to disrupt the TV and projection screen industries. Why would anyone spend $3,000 on a cutting-edge TV when the Vision Pro will give you a resizable screen that’s also portable and can easily display all of your content?

Quelle:

https://www.inverse.com/tech/apple-vision-pro-safari-web-browser-video

8. Januar 2024 0 comment

Knowledge Research

VR market keeps shrinking even as Meta pours billions of dollars a quarter into metaverse

Sales of VR headsets and AR glasses in the U.S. have plummeted nearly 40% in 2023, as of Nov. 25, according to data shared with CNBC by research firm Circana.
Meta has continued to burn through billions of dollars a quarter developing the metaverse, which CEO Mark Zuckerberg says is the future for his company.
The debut in October of Meta’s Quest 3 headset helped lift the VR market over the holiday period but wasn’t enough to make up for the rest of 2023.

Here comes Apple

Heading into 2024, the big wild card for the VR market is Apple

In June, Apple unveiled its Vision Pro mixed-reality headset, which is slated to hit the market next year at a starting price of $3,499.

The premium price suggests Apple is targeting early adopters, developers and companies as potential customers, VR developers told CNBC at Meta’s Connect event. VR enthusiasts are excited about Apple’s first headset, considering the company’s smashing success with consumer devices, and Vision Pro’s potential to integrate with products like the iPhone and iPad.

Apple didn’t respond to a request for comment.

The Vision Pro’s debut could also play a pivotal role in bolstering the fledging VR and AR market in 2024, according to research from IDC. In a September news release about the state of the market, Ramon Llamas, IDC research director, said, “Apple’s entry next year will bring much needed attention to a small market, but it will also force other companies to compete in different ways.”

Andrew Boone, an analyst at JMP Securities, said he was initially so impressed by Apple’s Vision Pro demos that he began to worry about Meta’s future in the market.

His thought at first was, “Apple was so far ahead that maybe Meta would just throw in the towel,” Boone said.

“I think my tone on that has changed,” he said. “I think the price was too high to actually get mass demand, so Zuck is going after a different version of this. Clearly, the Quest is more game focused.”

Boone says there’s “enough differentiation” between the Quest and Vision Pro devices that they can cater to different crowds, though he expects to learn a lot more about the VR market over the next 12 months.

Rolf Illenberger, CEO of German VR startup VRdirect, said companies are excited about the Vision Pro “because it’s Apple,” but there’s a perception that it’s more of a “lifestyle” device. Apple’s demos highlighted more entertainment-friendly uses like the ability to watch movies on a giant virtual display. Apple describes the Vision Pro as a “spatial computer,” capable of blending the physical world with digital content and visuals.

“That product is premium, so it also got people thinking about what does an ultra-premium experience look like and what are the use cases that arise from that,” said Circana’s Arnold.

High hopes for the enterprise

Illenberger sees the potential for Meta’s Quest 3 to make a splash in the enterprise for tasks like workforce training, onboarding and marketing. He noted that the device is $500 cheaper than the Quest Pro, which was released in 2022 as more of a business-focused device, and has many of the same features.

The consumer is more challenging. Aside from “early adopters and hardcore gaming kids,” Illenberger says, “there’s not enough convincing arguments to spend even $500 on VR.”

In the corporate VR market, Meta and Taiwan’s HTC are the leading suppliers of devices. Pico-branded headsets from TikTok parent ByteDance “are losing more and more ground,” Illenberger said. ByteDance has reportedly canceled the next version of its Pico headset and is instead shifting resources to another device more similar to Apple’s Vision Pro.

ByteDance didn’t respond to a request for comment.

When it comes to selling to businesses, Illenberger says Meta is starting to benefit from its name change in late 2021. He said that Zuckerberg’s rebranding has had a “psychological” impact on some companies who feel more more comfortable purchasing the devices without the tarnish of Facebook’s brand and the numerous associated data privacy scandals

“Rebranding the company to Meta was a genius move,” Illenberger said. “Not because he’s claiming the market for his company, but people more and more forget that Meta is in fact Facebook.”

Quelle:

Foto:

An attendee wears a Meta Platforms Inc. Oculus Quest 2 virtual reality (VR) headset at the Telefonica SA stand on day two of the Mobile World Congress at the Fira de Barcelona venue in Barcelona, Spain, on Tuesday, Feb. 28, 2023.Angel Garcia | Bloomberg | Getty Images

https://www.cnbc.com/2023/12/19/vr-market-shrinking-as-meta-pours-billions-of-dollars-into-metaverse.html

4. Januar 2024 0 comment

Hardware Knowledge

What is Apple Optic ID on the Vision Pro (and How Does it Work)?

Iris Scanning and Security with Apple Optic ID

Apple Optic ID represents just one of the latest features of the upcoming Apple Vision Pro headset. Ever since it announced its spatial computing headset in June 2023, Apple has begun gradually releasing more insights. Like with most Apple products, full details on the headset are scarce.

So far, we know the Apple Vision Pro will offer everything from intuitive mixed reality experiences to sensational spatial audio. Now, we also know the innovator is rolling out a new iris authentication system. The “Apple Optic ID,” solution will bring new levels of security to the Vision Pro.

But what is Apple Optic ID, how does it work, and what does it mean for the future of XR security? Here’s everything we know so far.

What is Apple Optic ID? The Basics

Apple Optic ID is the proprietary security solution for Apple’s latest mixed-reality headset, the Vision Pro. It was created with Apple’s “VisionOS” spatial computing operating system, using the latest biometric authentication technology.

According to the patent for Optic ID, the solution uses a combination of LED illuminators and infrared cameras within the Apple Vision Pro to map and analyze a user’s iris. Every person’s iris is distinctly different,, allowing for accurate authentication. Apple says the technology can even detect the unique attributes of identical twins, highlighting its exceptional accuracy.

This isn’t the first time Apple has introduced biometric technology into its devices. Already, the tech giant uses fingerprint sensors (Touch ID) on its smartphones and tablets. It also leverages Face ID technologies, which authenticate a user’s face using depth sensors and infrared dots.

However, Apple wanted to do something unique for its new mixed-reality headset. Rather than embedding touch ID into a button on the otherwise hands-free spatial computing device, they devised an all-new biometric model.

Notably, iris scanning tech isn’t entirely new in the authentication landscape either. These technologies are used in airports around the world to identify travelers. However, attempts to integrate iris scanners into devices and consumer technology haven’t always been successful. For instance, the iris scanner in the Samsung Galaxy S8 phone was hacked by researchers and fooled into identifying people based on photos rather than actual human eyes.

How Does the Technology Work?

Apple Optic ID uses iris scanning to facilitate the entire authentication process for Apple Vision Pro headset users. Similar to how you can use your fingerprint or face on your iPhone to log into your device, you can use the Optic ID technology to secure your XR headset.

Building in innovative eye tracking technology, Optic ID uses LED light exposure to take a 3D scan of your eye. The Vision Pro headset projects the light onto your eye and sends information back to an on-board computer system. This system then compares the scan against the scans you uploaded when configuring your Vision Pro device.

Notably, the iris scans you upload are stored securely in the Vision Pro “Secure Enclave,” which means the data never leaves your device. This means the experience is similar to the Face ID technology already on iPhone models.

All of your information is encrypted on your Vision Pro. Additionally, Apple announced at the WDCC event for 2023 that it’s also working on other security features to protect its users.

For instance, the operating system’s design ensures everything you look at through your headset remains private. Data will also be isolated in a separate background process, so websites and apps can’t see what you’re looking at. Only when you interact with content through your headset, using your voice or gestures, will data be communicated at a system level.

How Secure is Optic ID for Apple Vision Pro?

The Apple Optic ID feature demonstrates Apple’s commitment to making the Vision Pro more than a consumer-grade device. While you might not be too concerned about protecting your data on a personal XR headset, privacy is essential to XR in the enterprise.

Optic ID will ensure guests and other users can’t access the same information and tools as crucial employees within a business environment. It could even allow organizations to set up different access controls for specific staff members, allowing for greater data security.

But exactly how secure is Apple Optic ID? So far, the information available from Apple and its patent filings is limited. However, we know that Optic ID technology considers several parts of the iris, as well as the eyelids and eyebrows of each user, for a holistic approach to authentication.

Since each person’s iris is unique, the technology gives Apple an excellent way to deliver higher enterprise-grade security. Optic ID may perform better than Face ID, which Apple trained with over a billion images via its deep learning algorithm.

After all, while face ID tools may struggle to tell the difference between similar-looking people, Apple Optic ID can effectively distinguish between identical twins. Apple has configured its technology to rely on “3D depth data” to address other potential security issues.

This means you won’t be able to simply fool the system with a photo of an eye. As a bonus, since Apple secures all of the data held by Optic ID on your device, you can rest assured no one else can get their hands on your scan.

What Can You Do with The New Security Feature?

At present, Apple Optic ID is unique to the Vision Pro headset. So, you’ll need to wait until Apple eventually releases its new device to test it for yourself. Experts and industry analysts believe the Vision Pro should begin rolling out in early 2024. However, Apple has faced some delays, some of which could push production times back even further.

Primarily, Apple Optic ID is there to stop unauthorized people from accessing your expensive XR technology and the data stored on your device. It should help provide companies with a more secure XR experience, particularly if multiple team members share a headset.

However, the functionality of Optic ID doesn’t stop with authentication. Similar to the iPhone’s Touch ID and Face ID features, this biometric tech can accomplish other things. For instance, you can use it within VisionOS to:

Authorize purchases on the Apple App Store
Authorize payments with Apple Pay
Autofill saved passwords on apps and websites
Unlock hidden content on the Apple Vision Pro

This technology will likely also work with third-party apps on the Vision Pro when they launch, similar to Apple’s existing security measures. However, it’s uncertain whether users will need to implement additional authentication methods, like setting up a numeric passcode.

It’s also worth noting that the same technology powering Apple Optic ID will also help to enable highly accurate and precise eye tracking in the headset. This should ensure the “spatial computing” experiences on the Apple Vision Pro feel organic and natural.

Unlike other XR headsets, Apple isn’t introducing any controllers with its device, so it will rely entirely on eye and hand tracking capabilities to allow users to interact with virtual content.

Will Optic ID Appear on Other Devices?

If, like many business leaders, you’re excited about the security potential of Optic ID, you might be wondering whether it will be rolling out to other devices. After all, the Apple Vision Pro is an expensive piece of technology and one of the most advanced in the XR space.

Apple hasn’t released any information suggesting it will bring Optic ID to other tools for now. However, there’s every chance this technology and other aspects of the VisionOS operating system will trickle into new Apple devices.

If the tech giant continues to invest in the extended reality market, creating new smart glasses and even cheaper versions of the Vision Pro, these tools may feature Optic ID.

Additionally, Optic ID may appear on non-XR tools, like the new Apple iPhones, in the years to come. Apple has already demonstrated an interest in bringing spatial technology into its product portfolio.

The Future of XR Device Security

Apple Optic ID is more than just an exciting feature in a wide range of capabilities promised by the Apple Vision Pro. It’s an insight into the steps leading innovators are taking to make extended reality more secure and sustainable within the enterprise landscape.

High-level security features, access control settings, and compliance solutions will be essential for any business investing in the “enterprise-level” metaverse and XR tech going forward. Apple’s innovations in biometric security could inspire other developers to follow suit.

There’s a good chance we’ll see more business leaders implementing intelligent security features into XR and metaverse solutions as companies continue to invest in immersive tech.

Quelle:

What is Apple Optic ID on the Vision Pro (and How Does it Work)?

3. Januar 2024 0 comment

Knowledge Software/Apps

Video Offers a Glimpse of Vision Pro Hand-tracking & Occlusion Performance

A video posted by a Vision Pro developer appears to show the current levels of hand-tracking and occlusion performance that Apple’s new headset is capable of.

Apple Vision Pro, expected to launch in the next few months, will use hand-tracking as its primary input method. While system-level interactions combine hand-tracking with eye-tracking for a ‘look and tap’ modality, developers can also build applications that allow users to interact directly with applications using their hands.

Hand-tracking on Vision Pro can be broken into two distinct capabilities: ‘hand-tracking’ and ‘hand-occlusion’.

Hand-tracking is the estimated 3D model of the hand, its joints, and the position of the fingertips. This model is used to determine when objects are touched, grasped, and interacted with.

Hand-occlusion deals with how the system overlays your real hand onto virtual content. Rather than drawing a 3D model of a virtual hand into the scene, Vision Pro cuts out the image of your real hands to show them in the scene instead. Doing so adds another layer of realism to the virtual content because you can see your own, unique hands.

A developer building for Vision Pro using Unity posted a video which gives a clear look at both hand-tracking and occlusion.

In the video above, the real hands show the occlusion system in action, while the green dots show the estimated 3D position of the hand.

We can see that hand-occlusion is very good but not perfect; when surrounded entirely by virtual content and moving quickly, you can see some clipping around the edges of the hand.

Meanwhile, the actual hand-tracking position lags further behind the hand-occlusion, somewhere around 100–200ms behind. We can’t quite determine the true latency of Vision Pro’s hand-tracking however, because we only have a view of the occlusion-tracking hands to compare it to (which themselves will have some latency with regard to the real hands).

In this context, the video comes from a Unity developer conversation saying that their early experiments with Vision Pro showed very high hand-tracking latency, apparently in comparison to the latest Quest hand-tracking capabilities. Other Unity developers agreed that they were seeing similar latency on their own Vision Pro devices.

However, the video in question is a hand-tracking integration inside of a Unity app, which means the hand-tracking performance and latency may have additional factors relating to Unity that aren’t present when using Apple’s first-party Vision Pro development tools. And considering the headset hasn’t launched yet and these tools are still in development, we may see additional improvements by the time the headset actually reaches stores.

Quelle:

Video Offers a Glimpse of Vision Pro Hand-tracking & Occlusion Performance

1. Januar 2024 0 comment

Hardware

Apple Store Employees Start Vision Pro Training

Apple is preparing for the launch of its upcoming Vision Pro headset by scheduling two-day Vision Pro training sessions for retail employees.

The Vision Pro is inching closer and closer to a full launch with the start of training sessions for Apple Store employees.

As Bloomberg reports, Apple is now scheduling two-day classes to start sometime in January 2024. The pricey headset will reportedly only be available in stores so buyers can be guided through the setup process, which includes getting a custom fit on the device.

A proper fit is important because the Vision Pro is fairly heavy. Its size and weight could cause neck strain, which may be exacerbated by a bad placement. Apple is already working on this problem for the next generation of the Vision Pro, according to reports.

9to5Mac reports Apple Stores are now receiving „new equipment to assemble and box up“ Vision Pro headsets. Bloomberg says Apple expected the launch of the device to happen sometime in January, but it may not happen until March.

Apple took a similar approach the last time it waded into a new product category with the Apple Watch, which required in-store appointments when picking up the smartwatch.

The first-gen Vision Pro is being targeted at developers and enterprise customers, though anyone can purchase one. Those who spend that $3,499 will be able to tap into the „spatial video capture“ that rolled out today via iOS 17.2 for iPhone 15 Pro and Pro Max, which have cameras powerful enough to record videos in three dimensions.

Quelle:

https://uk.pcmag.com/vr-1/150078/apple-store-employees-start-vision-pro-training

Foto: Apple

21. Dezember 2023 0 comment

Examples

You Can Already Watch iPhone 15 Pro 3D ‚Spatial Videos‘ On Existing VR Headsets

You can now watch ‚Spatial Videos‘ recorded by iPhone 15 Pro on any VR headset.

Apple shipped the 3D video capture feature it calls ‚Spatial Video‘ in 17.2 beta 2 a few days ago. It captures two 1080p 30 FPS views from the phone’s wide and ultrawide cameras. These 3D videos are saved in the Apple HEVC Stereo Video Profile format, an implementation of MV-HEVC that the upcoming Apple Vision Pro headset will be able to play natively in a floating rectangular container.

Buying a $3500 headset won’t be necessary to view Spatial Videos, however. Apple publicly shared the specifications for the Apple HEVC Stereo Video Profile, and a developer has already built an iPhone app to let you convert it to a regular side-by-side video, which can be viewed in other headsets like Quest, Pico, Vive XR Elite, or any PC VR headset.

That iPhone app is called Spatialify. It’s not available on the App Store yet, it’s currently in TestFlight – effectively Apple’s equivalent to App Lab, if you’re unfamiliar.

One you install Spatialify, just choose your Spatial Video and tap ‚convert to Full Side-By-Side‘. You then need to transfer the file to your PC. You can use AirDrop for Mac, or a a third-party iPhone file transfer method for Windows.

For standalone headsets, the final step is to transfer the file to your headset via USB. You can view the video in the Files app on Quest or File Manager on Pico. When it’s opened, on Quest tap the ‚2D‘ icon in the bottom right and select ‚3D side by side‘, or on Pico, click ‚Switch Mode‘ then ‚3D SBS‘.

For PC VR headsets, you can use apps like Virtual Desktop Classic, DeoVR Video Player, Whirligig VR Media Player, or any other PC VR software which supports watching SBS 3D videos.

I tried watching some test clips iPhone 15 Pro owners have been uploading to social media in my Quest 3. While the sense of depth isn’t as pronounced as with dedicated 3D cameras that have a much wider stereo separation, it’s still clearly there, and it’s coming from a camera tens of millions of people have with them almost all the time.

The regular non-Pro iPhone 16 coming next year is rumored to be able to capture Spatial Videos too, so within a few years the creator base could rise to hundreds of millions. This raises the question: will this conversion process be required for long, or will companies like Meta soon support the MV-HEVC format natively?

Quelle:

https://www.uploadvr.com/watch-iphone-15-pro-spatial-videos-on-quest/

22. November 2023 0 comment

Knowledge Research

VisionDevCamp To Jumpstart Apple Vision Pro Developers In Early 2024

Shortly after Apple Vision Pro launch in early 2024 developers will gather for VisionDevCamp to work on app ideas for the new headset.

Long-time AR developer Raven Zachary has been working with others to organize iPhone developer hackathons since 2007. Now, he’s turned his sights to helping jumpstart spatial computing apps on Vision Pro with VisionDevCamp. The event is unaffiliated with Apple and will be held in the San Francisco area shortly after the U.S. launch of Apple Vision Pro early in 2024.

„In 2007, we organized iPhoneDevCamp to celebrate the launch of Apple’s new platform. In 2024, we’ll be staying true to this core focus for Apple’s newest platform and first spatial computer,“ Zachary told UploadVR. „The first year with a new development platform is always exciting, and we expect that excitement will be contagious at the event.“

Hundreds attended the first iPhoneDevCamp in July 2007, just a week after the U.S. launch of iPhone during a time when Apple only offered developers the option of making Web apps for the device. As Apple’s new development platform took shape in 2008 and beyond, the dev camp series grew too with iPadDevCamp and iOSDevCamp. Organizers say apps like Square, TestFlight, and Barack Obama’s 2008 election year app got their start at these events.

Registration isn’t open yet for VisionDevCamp, but there is a notification sign-up for when registration starts and also a Discord server to connect with others about the event.

„VisionDevCamp is a not-for-profit developer event focused on creating applications for Apple Vision Pro and visionOS,“ an announcement page explains. „Attendees are encouraged to develop native visionOS, Unity PolySpatial, and web applications during the event.“

Apple released its software development kit for Vision Pro in June and developers are currently building for the system using a visionOS simulator. Apple CEO Tim Cook confirmed in September that the headset is still on track to ship early in 2024.

Quelle:

https://www.uploadvr.com/visiondevcamp-vision-pro-developer-event-2024/

20. November 2023 0 comment