Mixed reality

On this page we will introduce some of the mixed reality features available on Varjo headsets.

These features are available for XR-1 Developer Edition.

Using mixed reality

Using the front-facing cameras on your headset, applications can combine virtual and real-world elements to create an immersive experience in mixed reality.

For the best possible image quality, calibrate eye tracking when you start using the headset. The headset will display content in the highest resolution in the area where you are looking.

Use the headset in a well-lit environment to get the optimal performance from your headset cameras.



Real world view

With Real world view, you can see the physical environment around you through the mixed reality cameras on your headset. You can quickly switch between your application and the real world while wearing the headset.

Real world view can be accessed both from the headset and in Varjo Base. On the headset, a long press of the System button will display Real world view in the virtual environment. To launch from Varjo Base, navigate to the Tools menu and select Real world view.

To exit, long-press the System button again or disable Real world view from the Tools menu in Varjo Base.



Varjo Markers

Varjo Markers are used to indicate the position of virtual items in your mixed reality application. Instructions for printing Varjo Markers can be found on our Developer portal.

The status area in Varjo Base will show the markers that have been recognized by the headset. If the headset has recognized a large number of markers, you can see their details by clicking on the status area to expand it.


Markers are processed by your headset only if the current application supports the use of  Varjo Markers.

If the markers have not been recognized by the headset for a period longer than two minutes, they will be automatically cleared from the status area.

Please note: The markers have unique IDs. You should not use the same marker more than once in any given environment.



Chroma key

The chroma key feature allows you to control where a viewer sees virtual content in the mixed reality environment. When chroma key is in use, a predefined color in your physical environment is replaced with virtual content.


Preparing your environment

Chroma key works best in good, soft lighting. Make sure the chroma key surface is evenly lit and avoid reflective objects in your environment, as the chroma key background can become visible in these places. Try to use neutral-colored lights with identical color temperatures.

Depending on your intended use, we recommend that you choose a chroma key color that is clearly distinct from human skin tones. Green and blue are typically good choices. Chroma keying requires a bright color tone as its target, and therefore black, white, or gray will not work.

Chroma key can be used either as a background for the scene or on smaller individual surfaces. When using a colored canvas as backdrop, make sure not to leave wrinkles, creases, or shadows on the surface. It helps to leave some distance between foreground elements and the background to avoid shadows falling on the canvas. When placing smaller individual elements into the scene, ensure that they are all the same color and lit similarly.


Chroma key settings

In Varjo Base, the chroma key feature is found under the System tab.

Open Define chroma key color to pick and adjust the color used for chroma key. Enable Use chroma key in all applications to force all applications to use chroma key and the mixed reality cameras. Areas in the chroma key color are replaced with application content.


You may see one of the following messages in the System tab:

Configuration locked by another application. Another application is using the chroma key configuration. All chroma key settings are disabled. The settings will be released once the application is closed or is no longer using the configuration.
Chroma key set by application. Sampling a new color will erase the current chroma key. Another application has set the current chroma key configuration. Editing the configuration will replace the current one.


Picking a chroma key color

To pick a chroma key color, navigate to the System tab and open Define chroma key color.

You can preview the current chroma key configuration using the Preview chroma key setting. The chroma key mask is visualized using the color you select in the drop-down list.


Start Pick colors with headset to select a chroma key color using the mixed reality cameras on your headset. While holding the headset, point it at your target color and use the Space bar or the Application button on your headset to pick color samples. The color you pick is replaced with the preview color in the headset view.

We recommend you pick colors from multiple points in the environment to account for differences in lighting and surface color. The color picker is additive: each selection increases the color range in use. Select Stop to quit the color picker.

Reset will return the original color values. You can also load a color configuration or save the current one.

Your chroma key color can be fine-tuned under Adjust chroma key color. The sliders for Hue and Hue tolerance adjust the chroma key color tone and the tolerance for color variation. Saturation tolerance and Value tolerance adjust the tolerance for bright and dark areas respectively.

Accept the current chroma key color and return to the System tab by clicking Done. Revert to the previous configuration by selecting Cancel or closing the settings menu.