How Hand Tracking in VR Unlocks Enterprise Use Cases
For professional XR users, the improvements in headset resolution led by Varjo have been a game changer. Parallel to this visual revolution, we’re also seeing a revolution in interaction in enterprise XR.
Guest post by Matt Tullis, VP of XR, Ultraleap
Guest post by Matt Tullis
VP of XR, Ultraleap
As the adoption of AR and VR technology accelerates in both enterprise and consumer markets globally, the limitations of traditional handheld controllers are becoming ever clearer. Hand tracking, eye tracking, voice control, and the use of physical controls or props in combination with video passthrough are all now augmenting or completely replacing hand-held controllers.
Arguably, the most demanding use cases are to be found in enterprise XR – with applications such as simulation and training for pilots, or 3D design reviews and collaboration for car design.
With the launch of their XR-3 and VR-3 product range, Varjo demonstrated their leadership position in enterprise XR. Among other industry firsts, with the XR-3 and VR-3 Varjo is first to market with Ultraleap’s fifth-generation hand tracking software – Gemini.
The Ultraleap Gemini hand tracking solution comes fully integrated into the Varjo XR-3 and VR-3 headsets and enables the creation of XR enterprise applications that:
- Are more approachable, easier to learn, and lower friction (particularly for users with little or no experience of hand-held controllers)
- Deepen the realism of complex simulations
- Improve communication, trust, and social cohesion by enabling more lifelike avatars
Let’s look at the value this adds in three specific enterprise use cases.
Immersive design workflows: How approachable, low-friction interaction improves productivity
Varjo headsets speed up design processes and reduce costs by bringing globally distributed design teams together in a photorealistic virtual space. Being able to interact using only your hands widens participation in these processes. It makes them more streamlined, collaborative, and efficient.
Hand tracking enables anyone to engage with a design workflow immediately and intuitively – even if they have no experience of VR or gaming. They can provide comments, interact with, and move components around, just using their hands. In augmented and mixed reality design workflows, hand tracking allows users to switch fluidly between interacting with real and virtual objects.
Hand tracking enables key decision-makers without the time or inclination for a lengthy onboarding process to be brought into workflows more effectively. A time-poor C-suite executive, a demanding customer, or a subject matter expert commenting on a prototype can confidently take part in productive virtual design sessions. This improves decision-making and reduces the need for business travel.
Training and simulation: High-fidelity interaction deepens realism
The realistic simulations created by Varjo’s human-eye resolution visual fidelity are further enhanced when paired with high-fidelity interactions – i.e. interactions that closely mirror the way we interact with objects in the real world.
Our sense of immersion is dependent on both the visual fidelity of virtual objects and the quality of our interactions with those objects. Seeing a virtual body that moves and interacts in the same way as your own body induces a strong sense of agency.
Interaction in enterprise simulations is complex and multi-faceted. Hand tracking adds real value in scenarios such as:
- Users are being situationally trained to interact with their hands, such as a flight attendant closing overhead bins prior to takeoff.
- Combinations of real physical objects along with virtual items are being used and learners need to switch fluidly between the two – such as pilot training in VR along with real physical instrumentation.
- Specific hand motions are measured for accuracy or timing, such as in surgical training.
Remote collaboration: How hands speak and why hand tracking improves communication
Every culture gestures with their hands while they talk. There is a significant body of research showing that hand gestures reflect a speaker’s thoughts (often subconsciously), change the way listeners interpret what is being said, and even affect fundamental aspects of how words are understood.
Adding hand tracking to XR remote collaboration enables the creation of avatars with hand movements that mirror the gestures the real person is making as they talk. By expanding remote collaboration capabilities to include this key element of non-verbal communication, hand tracking enables richer and more lifelike communication between people working in virtual spaces.
This reduces miscommunication, builds trust, and increases social cohesion. All these are strongly correlated with better team and business performance.
All indications are that remote working will permanently increase post-pandemic. Solutions that enable remote teams to be cohesive and highly effective in the long term offer the potential for exponential cost savings, without a corresponding trade-off in performance.
Developing with hand tracking on the Varjo VR-3 and XR-3
With Varjo XR-3 and VR-3, developers can use the latest generation of the world’s most reliable and robust hand tracking software. Gemini unlocks powerful interactions that will define the next generation of immersive computing.
Developers working on hand tracking applications are supported by Ultraleap’s detailed XR design guidelines and design tools. This includes Unity and Unreal plugins as well as Ultraleap’s award-winning Interaction Engine. They enable unprecedented natural interaction with virtual interfaces, wearable interfaces, curved spaces, and complex object physics.
Using hand tracking as an input modality requires a change in mindset if you’re used to developing for controllers, but for enterprise XR the potential rewards are huge.
