Hand tracking with Varjo headset

Varjo’s VR-2 Pro is coming with built-in hand tracking technology from Ultraleap. It allows user to use hands without any physical controller and reach out with hands into VR world. Motions like pinching, grabbing, and interaction with objects add new level of immersion you could accommodate in your applications.

 

Setup

In order to enable hand tracking module, you must download Ultraleap SDK for Varjo from our downloads page. It is important that you download UltraleapSDK from Varjo website, as we provide software version that was specifically tailored for VR-2 Pro and was stripped down from all the unnecessary components.

When both the Varjo and the Ultraleap SDK are installed, and the VR-2 Pro is correctly connected you should see Ultraleap icon turning green in Windows dashboard:

 

Developing with hand tracking

You can develop hand tracking support for Unreal engine, Unity, or natively. Refer to the Ultraleap documentation to follow up to date guides on integration and best practices of using hand tracking in VR.

When you start development of hand tracking make sure to set offset for the hand position first. Since the head tracking point for the headset and hand tracking point for Ultraleap are not the same, offset needs to be defined. Offset is following (depending on your engine of choice, you might have different scale):

Y: -0.025734 m
Z: 0.068423 m
X tilt: 5.0°
OR
Y: -25.734 mm
Z: 68.423 mm
X tilt: 5.0°

 

Unity Ultraleap Plugin installation

If you want to add hand tracking to your Unity project follow these steps:

  • Close your Unity project
  • Download Varjo VR-2 Pro Ultraleap SDK for Unity from Github
  • Copy Assets folder from Ultraleap SDK for Unity to your project’s folder
  • Start your Unity project
  • Select VarjoCamera and Add new component to it called “Leap XR Service Provider”

  • Set the Device Offset Mode to Transform and drag HandTrackingOffset game object to Device Origin (HandTrackingOffset can be found under VarjoCamera)

  • (Optional) Enabling “Update Hand In Precull” might improve latency in some cases

For adding 3d models of the hands refer to the Ultraleap documentation.

 

Unreal Ultraleap Plugin installation

To add hand tracking to your Unreal project follow the instructions on the Varjo VR-2 Pro Leap Motion SDK for Unreal Engine Github page.