Tag

Fingertracking

A Guide to Comparing XR Hand and Eye Tracking Technology in 2022

Choosing the right tracking technology

Hand and eye tracking solutions represent one of the most significant innovations in the XR space. Designed to boost the immersive experience of stepping into extended reality, these tools make the process of interacting with virtual content feel more natural.

On its own, the gesture recognition market is expected to reach a value of $70.18 billion by 2030, while the eye tracking market is expected to grow at a rate of 22.4% from 2022 to 2030. The question for companies leveraging the power of XR for training, innovation, and collaboration, is how do you choose the right hand and eye tracking tech?

Let’s look at some of the most important points you’ll need to consider when buying XR solutions with embedded hand and eye tracking features in 2022.

Step 1: Set Your XR Goals

The first step in choosing the right hand and eye tracking solutions for your business needs, is setting some goals. Tracking technologies can serve a range of purposes in the corporate landscape. You could use your eye tracking technology to improve the security of your collaborative environment, with biometric scanning for everyone who uses a headset.

Another option could be to enhance the way your teams collaborate. With hand and eye tracking solutions built into VR, it’s easier for teams to interact with virtual content like digital twins. Teams can even use the same technology to enhance their training experiences, develop muscle memories, and learn new skills. Some of the goals you might set include:

Improved insights: Hand and eye tracking solutions can help you to collect all kinds of data. You can gather useful insights into how your employees interact with virtual content to improve training opportunities. Alternatively, you might collect information about your customers, looking at how they interact with products and building insights into sentiment.
Better technology efficiency: Certain tracking tools in the XR landscape can help to make your tools more efficient. Solutions like foveated rendering, for instance, reduce GPU load and power consumption significantly. By focusing on enhancing a specific portion of a virtual image, rather than the whole experience, you reduce business costs.
Deeper immersion: One of the biggest benefits of both hand and eye tracking technologies, is the potential to create more immersive experiences. If you want the XR environment to feel completely natural to your employees during work and training sessions, the right tracking solutions can help significantly.

Step 2: Learn about the Tracking Landscape

Once you know what you want to accomplish with your eye and hand tracking technology, the next step is learning about the different tools on offer to help you achieve your goals. Both eye and hand tracking tech come in a range of different flavors.

Most eye tracking technology uses near-infrared solutions and high-resolution cameras to track gaze direction. The underlying concept is known as “PCCR”, or Pupil Center Corneal Reflection. Eye tracking devices can also come in different designs. Screen-based solutions are built into glasses and headsets, while webcam and remote tracking is a slightly more old-fashioned alternatively.

The functionality of your hand tracking technology can also differ depending on the kind of tools you choose. Most solutions start with sensors which are either attached to your headset, or placed in locations around the user. These tools capture information using cameras, while software in the XR environment makes sense of your movements.

If you’re not sure what kind of hand and eye tracking solutions are best for your needs, it’s worth chatting to a leader at an XR business for more insights. Many vendors will be able to give you a run-down of how their technology works. More importantly, they should be able to provide demonstrations so you can get a feel for the solution.

Step 3: Consider Whether AR, MR, or VR is Best

Based on your use cases, and your knowledge of hand and eye tracking solutions, you’ll be able to decide what kind of XR innovation makes the most sense for your needs. VR solutions with hand and eye tracking are generally more likely to appeal to companies looking to improve collaboration.

With VR experiences featuring both eye and hand tracking technologies, you can make it easier for team members to interact with both virtual content, and eachother. Some tools will even improve the bonding potential between team members by recreating facial expressions, and making the overall interaction feel more human.

AR and MR solutions will appeal to scenarios where your team members need to connect with both the physical world, and virtual content at the same time. Engineers who need to be able to flip through virtual manuals hands-free while working on a machine will benefit from MR innovations.

AR tools with eye tracking technology can make it easier to feed video information back to specialists in an office environment from employees in the field. This can allow for the delivery of useful guidance in real-time.

Step 4: Prioritize a Customisable User Experience

Finally, it’s worth remembering the needs of each individual user and company are likely to vary when it comes to hand and eye tracking in XR. The right solution for your needs should be perfectly suited for the kind of use cases you’re trying to address.

For instance, you might want to look for a VR headset with all the tracking systems you need already built into the headset itself, to reduce the amount of additional sensors you need to set up in a small office space.

You may need to consider the comfort of your employees if they’re going to be using your XR innovations for extended periods of time. A flexible and lightweight headset will likely allow for more comfortable working experiences, particularly for employees on the move.

Many vendors will be able to discuss your needs with you and offer customizable solutions with software and hardware elements you can tweak to suit your requirements. Take the time to discuss your requirements with each vendor before you start investing.

Quelle:

A Guide to Comparing XR Hand and Eye Tracking Technology in 2022

24. Juli 2022 0 comment

Examples

Manus Begins Pre-orders for Latest VR Gloves, Quantum Metagloves

VR glove creator Manus has launched pre-orders for its latest device, Quantum Metagloves, an input solution targeted at enterprise users. The Netherlands-based company is looking to offer up more precise finger tracking than not only its past VR gloves, but also optical trackers current used in the industry.

Manus launched pre-orders for Quantum Metagloves last week, stating that expected shipping begins sometime in early Q3, which could likely take place in the September timeframe.

The enterprise-focused gloves start at $9,000 (€7,500), and include a perpetual license and a 2-year warranty. That pricing reflects the base unit without 6DOF positional tracking though, but Quantum Metagloves feature optional modules that hook into Xsens, OptiTrack, and SteamVR tracking standards.

If you haven’t seen our hands-on with the Quantum Metagloves, here’s a quick rundown of how they work. Unlike past Manus gloves, the company’s latest uses a magnetic base placed on the back of the palm. Each finger is tracked magnetically via corresponding modules snuggly placed on the each finger’s tip.

While this doesn’t include haptic feedback, the company told us at GDC 2022 that it was working on a haptic version of the Quantum Metagloves, which would include per-finger haptics to enhance immersion in VR.

At any rate, the gloves offered some promising results when it came to beating occlusion and demonstrating fine finger movements. Make sure to check out all of our early impressions from GDC 2022 in March to see it in action and find out more.

Here’s a look at what comes in the box:

A set of Manus Quantum Metagloves
A perpetual Manus Core Pro license
Wireless Manus Dongle
Three full sets of finger caps (1/2/3)
One full set of finger tapes
Three sets of liner gloves (S/M/L)
Timesync cable
Manus Travel Case

Manus is also running a trade-in discount program which will allow owners users of a number of past products to get Quantum Metagloves for €2500 / $3000 off retail price. These include owners of Prime II, Prime II Haptic, Prime II Xsens, Prime X, Xsens Gloves by Manus, and OptiTrack Gloves by Manus. Visit Manus to learn more about trade-ins and pre-orders.

Technical Specifications

Signal Latency ≤ 5ms
Finger Sensors Absolute position, 3 axis rotation
Sensor sample rate 120Hz
Battery life 4 hours (swappable)
Weight 70 grams
Wireless communication High Performance 2.4 Ghz Protocol
Wireless range Up to 15 meters
Wired communication USB Type-C
Finger attachments Finger caps, finger tapes, liner gloves
Glove Textiles One size fits all (washable)

Quelle:

Manus Begins Pre-orders for Latest VR Gloves, Quantum Metagloves

17. Mai 2022 0 comment

Hardware Knowledge

Manus Quantum Metaglove Demonstration – GDC 2022

Latest Manus VR Gloves Promise New Levels of Finger Tracking Accuracy

At GDC 2022 this week, VR glove creator Manus revealed its new Quantum Metagloves which the company says delivers significantly more accurate finger tracking than its prior solutions. Though priced for enterprise use, the company says it one day hopes to deliver the tech to consumers.

Manus has been building motion gloves for use in real-time VR and motion capture for years now, with prior offerings being based on IMU and flex-sensor tracking.

The company’s latest product, the Quantum Metagloves, moves to a new magnetic tracking approach which purportedly offers significantly more accurate finger tracking, especially when it comes to self-contact (ie: fingers touching other fingers or the palm of the hand).

Revealed at GDC 2022 for the first time, Manus showed off a demo of the Quantum Metagloves using a realistic real-time hand model that mirrored the wearer’s finger movements. Though the gloves are designed to work in conjunction with 6DOF tracking (via a SteamVR tracker or other motion tracking tech), the GDC demo didn’t employ 6DOF (which is why the visualization of the arm rotates in place). The latency reflected in this setup is also purportedly not representative of the actual tracking latency.

The Quantum Metagloves have a magnetic base positioned on the back of the palm while each finger has a module on the tip that is sensed within the magnetic field. Manus says this means the gloves can detect absolute finger length and width (once calibrated), which enables more accurate hand-tracking when combined with an underlying skeletal model of the hand that is scaled dynamically to the user.

In the video I asked the demonstrator to make a handful of different poses. Indeed, finger-to-finger and finger-to-palm contact looked impressive with no obvious clipping or stuttering. The company told me the demo wasn’t specially programmed to make clipping impossible and that the behavior was purely thanks to the positional data of the sensors which was described as “very clean” compared to alternative approaches to finger tracking.

Manus says the Quantum Metagloves are unique in this way, as other finger tracking technology tends to break down in these sorts of close-contact and self-contact scenarios, especially when both hands are near or touching each other. Even expensive optical tracking systems (with markers on the tips of each finger) can be foiled easily by self-occlusion or one hand occluding the other. Similarly, purely IMU-based finger tracking is prone to drift and requires regular recalibration.

But magnetic tracking is by no means perfect. In other magnetic tracking systems we’ve seen challenges with latency and electromagnetic interference.

Manus admitted that holding metallic or electronic items could throw off the tracking, but says it worked hard to ensure the gloves don’t interfere with each other; up to eight gloves can be active near each other without interference issues, the company says.

While self-contact looked generally quite good with the Quantum Metagloves, other poses didn’t fare quite as well—like a completely clenched first. The demonstrator suggested this would be improved easily with a more robust calibration process that included similar poses; whereas they say the calibration used for the demo at GDC was designed to be quick and easy for purposes of the show.

While the finger tracking did look great in many of the demos I saw, some of the other demo gloves on display showed much less accuracy. This was chalked up to “calibration,” though a big question for such systems is how much said calibration drifts over time and whether the periods between recalibration are practical for a given use-case.

In any case, use-cases will be deeply constrained by price; Manus says a pair of the Quantum Metagloves will cost $9,000, with pre-orders opening in April and shipments expected by the end of Q3. The company says it also plans to launch a haptic version of the Quantum Metagloves which will include per-finger haptics to enhance immersion in VR.

Manus maintains that it would like to bring its gloves to consumers one day, but says the number of custom parts and manufacturing makes it difficult to get the price down to a reasonable level.

Quelle:

Foto: Road to VR

Latest Manus VR Gloves Promise New Levels of Finger Tracking Accuracy

6. April 2022 0 comment

Knowledge Research

‘Hand Physics Lab’ Review – Putting the Fun in Frustration

Hand Physics Lab is all about hand-tracking, and this over 80-level game aims to throw you head first into a whimsical series of seemingly simple tasks. Quest can offer mixed results when it comes to hand-tracking fidelity though, so if you’re not careful, it can be extremely frustrating for all the wrong reasons.

Gameplay

Messing around with hand-tracking is really fun the first couple times you try it, and seeing my twiddling fingers represented in VR really served up some of the first ‘wow’ moments for me. Despite the promise of controller-less interactions, hand-tracking on Quest hasn’t been adopted by game developers across the board for a few reasons, a major one being that it offers lower fidelity input in comparison to Touch.

Note: If you haven’t given hand-tracking a spin on either Quest headset, you just need regular indoor lighting for optimal results. I found closing my shades and turning on all my interior lights provided adequate tracking. Still, it’s not uncommon to see your virtual hands completely wig out, or not track all of your fingers correctly—it just doesn’t feature the rock-solid tracking and button inputs of Touch.

Because Hand Physics Lab has virtualized your hands into physics-based objects, it’s less about directly interacting with objects and more about carefully miming actions and piloting a pair of decidedly much less capable limbs. It feels a bit like playing Surgeon Simulator at times, except the fidelity of input is constantly changing. It’s hard to predict when hand-tracking will fail and that lightbulb you tried to screw in for the past five minutes gets launched into the stratosphere.

Hand-tracking funkiness is an Oculus problem, but Hand Physics Lab seems to be leaning into it with mixed results. Even in the best case, this is the level of jitter you should expect. Keep an eye on the left hand as the fingers and whole hand pulse. In the worst case, your hand won’t work. Like at all. https://gfycat.com/ifr/altruisticpiercingarrowana

Ok, so maybe it’s comically difficult by design because the hand-tracking control scheme isn’t precise. Hand Physics Lab also offers Touch controls, but that feels a bit like cheating because it lowers the skill ceiling so low that it’s not really even worth playing at that point. The creator assumes you’ll want to play with hand-tracking, so I did.

And like Surgeon Simulator, it definitely results in some madcap things like getting your virtual arm stuck in a tube while your physical hand moves through space unimpeded. When it’s working well enough it can be really fun, but when you’re not sure why hand-tracking is failing and it just does, playing the game can feel like trying to punch someone in your dreams. Totally ineffectual for no discernible reason.

Image courtesy Dennys Kuhnert, Holonautic

In Hand Physics Lab you run through about 80 tasks, most of which are regular things even chimps are expected to be able to do: Press these buttons. Sort these colored blocks. Stack these colored blocks. Paint these eggs different colors. And although only some that can be considered puzzles as such, there are a few really interesting tasks in the bunch that take some patience and attention to accomplish, like having to snatch target blocks out of a tumble dryer, or using a sort of grip-based telepathy to guide blocks through a series of tubes.

After completing the final sequential task, I was pretty much done with Hand Physics Lab though. Its more of a quirky, bright smorgasbord of tech demo stuff than a full-featured game as such. I say that full-well knowing that there’s actually a fair bit of content here; you play sequential levels to earn stars in order to unlock sandbox activities.

Putting hand funkiness aside, I wished offered bigger and better puzzles. There was a glimmer of some really interesting tasks that very briefly ramped up to real puzzle territory, and I would have loved to see those teased out more.

Immersion

At risk of repeating myself, I’ll put it this way: hand-tracking funkiness really inhibits the promise of this game. If hand-tracking on Quest were hypothetically rock solid, and this game intentionally made it tremble and wig out for comedic effect, you might appraise it differently.

Anyway, I typically would play around 10 minutes at a time, but after playing for an hour straight, I got extremely frustrated with the game (read: ~~I rage quit~~ I tactically regrouped) and I had to write down my immediate impressions:

I feel like I have a wasting disease writing this. I just played Hand Physics Lab for an hour straight, and now my hands feel like weak meat puppets.
I’m having to concentrate very hard to coordinate my hands to type this correctly, and my arms feel extra heavy and useless.

The game had screwed with my proprioception, or the sense that you know where your limbs are at any given time without looking at them. More accurately, I was actually experiencing a ‘body transfer illusion’, which can be replicated by hiding a person’s hand from sight and replacing it with a fake one. Stroking both the fake hand and the subject’s real hand with a feather actually creates something called ‘proprioceptive drift’, which essentially makes the person feel like the rubber hand is their actual hand.

https://gfycat.com/ifr/illbetterkiwi

Because I was so intently staring at my virtualized hands the entire time, which don’t move perfectly in concert with my physical hands, I had actually internalized that my hands had become less effective at fine coordination tasks like typing or picking things up afterwards when I left the headset. Revisiting the game for shorter periods made this less apparent, but it’s safe to say I won’t be playing for extended sessions anytime soon. I took a break after that, and my arms eventually felt normal again, which was a relief.

Smaller bites and slower, more intentional movement can lessen this, but just don’t go in expecting to interact normally.

Comfort

Hand Physics Lab can be played seated, standing, and allows for a variable amount space. Besides the intensely weird body illusion stuff, it ranks among the most comfortable VR experiences since all locomotion is physical and not artificial such as smooth turning, snap-turning, or teleporting.

Quelle:

‘Hand Physics Lab’ Review – Putting the Fun in Frustration

5. April 2021 0 comment

Knowledge Research

Hand Tracking Wristband Brings More Detailed Hand Movements To VR & AR

A wristband with four thermal sensors promises a more detailed hand tracking experience.

The minds over at Cornell University’s SciFi Lab, along with help from the University of Wisconsin-Madison, have come up with a unique wristband device that uses four small (9.30mm) synchronized thermal cameras to deliver more precise VR and AR hand tracking experiences.

The device is called FingerTrak and looks a little bit like a souped up version of Spider-Man’s web shooters. Instead of firing webs, however, this device captures your hand movements and then transfers that data to a tethered Raspberry Pi-powered device connected to a computer. That computer then runs a deep learning model which can predict all 20 finger joint positions.

Each thermal camera tracks the movements of the top, bottom, left, and right part of your hand and then meshes each image into a single stitched thermal image in real-time. Because FingerTrak uses thermal images to track your hand movements, the system is capable of more precise hand tracking when compared to options such as the Leap Motion Controller or the Oculus Quests.

To announce their advancements in hand tracking technology, the SciFi Lab released a white paper along with a video detailing their work on its thermal camera tracking device. In the video, you get a POV of the user’s bare hands as the individual handles day-to-day activities such as flipping through a notebook and writing a journal entry, holding a cup of coffee, and picking up their mobile device.

Even subtle hand movements are capable of being tracked. Low-resolution (32 x 24) thermal cameras are able to estimate your bare hand posture by tracking the outline of your hand silhouettes.

The video also shows how FingerTrak gives you the ability to control a robotic hand in real-time. As the user moves their fingers, makes a fist, and closes and opens their hands, the robotic hand mirrors each and every movement in real-time. This kind of technology could prove incredibly useful in a variety of dangerous situations, such as defusing a bomb, exploring dangerous environments, or even providing basic at-home care remotely.

According to Sci-Fi Lab, however, there’s still plenty of work to be done before their product is ready for mass production. Similar to most other forms of consumer hand tracking technology, FingerTrak is unable to provid any form of haptic feedback, making it difficult to complete tasks that rely on you being able to “feel” the world around you. Another issue is that FingerTrak has a small angular error rate that fluctuates between 6.46 to 8.06 degrees. While that may not sound like a big deal, this can make a world of difference during activities that require precision, such as cutting specific wires on a bomb.

If you’re interested in learning more about FingerTrak, The SciFi Lab will be presenting its work at the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing in September. You can also access their white paper here.

Quelle:

Image Credit: Sci-Fi Lab

Hand Tracking Wristband Brings More Detailed Hand Movements To VR & AR

27. Juli 2020 0 comment