Eye tracking with Varjo headset

Varjo headset comes with the 20/20 Eye Tracker, our integrated eye tracking functionality. You can use eye tracking in your application, and you can log the information about gaze for analytics. Eye tracking can also be used for interacting with content: you can use it for selecting objects or prompting additional information about a specific object by simply looking at it.

 

What is gaze

Gaze starts from the gaze origin, which is the eye, and follows the gaze vector, which is the direction of the gaze. Normalized gaze vector is calculated. It is important for you to understand this concept in order to process eye tracking data while developing for Varjo Headset.

 

Eye Tracker Specifications

Varjo headsets contain two eye tracking cameras – one for each eye.

  • IPD range – 61–73 mm
  • Gaze camera resolution – 1280 x 800 px per camera.
  • Gaze tracking frequency – 100 Hz
  • Eye tracking accuracy – 1°
  • Eye tracking precision – 0.2°

 

Using the Eye Tracking data

You can get following eye tracking data from Varjo API: 

  • leftEye [varjo_ray]
  • rightEye [varjo_ray]
  • gaze [varjo_ray]
  • focusDistance 
  • stability
  • captureTime
  • leftStatus 
  • rightStatus
  • status 
  • frameNumber 
  • leftPupilSize 
  • rightPupilSize 

 

varjo_ray contains information about eye position coordinates origin(x, y, z) and for the direction of vector forward(x, y, z) where the user is looking. varjo_ray info is recorded for left eye (leftEye), right eye(rightEye), and combined value (gaze). Information about gaze is presented in the left hand coordinate system and is relative to the head pose.

focusDistance is the distance between the eye and the focus point. It is a value between 0 and 2 meters.

stability is a value between 0.0 and 1.0 specifying the stability of the user’s focus. 0.0 means not stable, 1.0 means stable.

captureTime is the timestamp of when the data was recorded in nanoseconds.

leftStatus and rightStatus is a value for each eye being:

  • 0 – eye is not tracked and not visible (e.g. eye is shut)
  • 1 – eye is visible but not reliably tracked (e.g. saccade or blink)
  • 2 – eye is tracked but quality compromised (e.g. headset has moved after calibration)
  • 3 – eye is tracked

status is a value representing the status of eye tracking in the Varjo headsets, being:

  • 0 – data is not available, user is not wearing the device or eyes cannot be found
  • 1 – user is wearing the device, but gaze tracking is being calibrated
  • 2 – data is valid

frameNumber is a unique identifier of the frame when the data was recorded.

leftPupilSize and rightPupilSize are the values for the size of the pupil between 0 and 1 calculated from knowing largest and smallest possible pupil size on Varjo headset.

 

Developing with 20/20 Eye Tracker

Before trying to launch any demo with eye tracking, make sure to enable the Allow eye tracking in the Varjo Base on the System page.

While developing, keep in mind that every time the headset is taken off and put back on it must be re-calibrated, even if it is the same person using it. This is needed because the position of the headset on the user’s face may not be the same every time when wearing the headset. You can manage calibration directly from your application. For detailed instructions, refer to the documentation of the engine of your choice.

 

Fast calibration

Two parameters are passed for the tracking – “GazeCalibrationType” can be either “Legacy” or “Fast”. Legacy is the calibration that is currently used by default which has long calibration time. Fast calibration is new 5-point calibration which requires much less user time to complete. However, it is currently in the Beta state and it’s validation is ongoing, so for proven results rely on Legacy calibration.  Second parameter passed is “OutputFilterType”. It defines if you want to receive smoothed tracking data which is easier to read, or raw data. For raw data pass key “None”, for processed data pass “Standard”.

In order to request specific data call RequestGazeCalibrationWithParameters and pass GazeCalibrationParameters as a parameter.

parameters = new VarjoPlugin.GazeCalibrationParameters[2];
                parameters[0] = new VarjoPlugin.GazeCalibrationParameters();
                parameters[0].key = "GazeCalibrationType";
                parameters[0].value = calibrationType == CalibrationType.LEGACY ? "Legacy" : "Fast";
                parameters[1] = new VarjoPlugin.GazeCalibrationParameters();
                parameters[1].key = "OutputFilterType";
                parameters[1].value = outputFilterType == OutputFilterType.STANDARD ? "Standard" : "None";

VarjoPlugin.RequestGazeCalibrationWithParameters(parameters);