Talk to sales

Unlocking Mixed Reality with Chroma Key: Expert Tips for Creating Your Chroma Studio

February 2, 2023
by Mike Carmody, Solutions Engineer, Varjo
|
XR/VR tips and tricks

Did you know that you can leverage real-time green screening in your immersive applications? Want to achieve a seamless blend of virtual and real elements without coding skills? Read on to learn how to unlock the full potential of mixed reality by utilizing chroma key with Varjo’s mixed reality technology.

Contents:

Chroma Key Pros & Cons: When is the Green Screen the Best Choice  Tips and Techniques for Building Your Own Chroma Space

Mixed Reality Chroma Key Space
Chroma keying, also known as “green screening” or “blue screening,” is a technique used in video production to superimpose one image or video over another by removing a specific color, usually green or blue, from the background of the top layer. This allows the background to be replaced with a different image or video, creating the illusion that the subject in the top layer is in a different location or environment. With this technique widely-used in the broadcasting and film industry, Varjo revolutionized the game by being the first to offer real-time chroma keying for mixed reality devices.

So what are the benefits and essentials for using this technique in immersive experiences? In this Q&A, we will explore the steps and tools necessary for building your own chroma space, and how you can make the most of this powerful technology to enhance your XR experiences.

Pros and Cons of Chroma Keying: When is the Green Screen Your Best Choice?

Varjo headsets offer built-in chromakey capabilities, making it possible to use chromakeying with any software run through the headset. However, there are specific kinds of experiences that work particularly well with this setup, creating a feeling that you’re stepping into a completely other reality.

Why should I use chroma keying?

Chroma keying enables very high precision occlusion of virtual objects by real objects. Therefore, solutions where the most important feature is the ability of the user to accurately see real objects best take advantage of chroma keyed environments.  A “real object” could be something handheld, furniture or even another person in the room — everything real will be visible “on top of” the virtual elements.  As we’ll discuss further down, you can also combine chroma keying with other Varjo features to ensure that you are still able to have physical and virtual elements interact appropriately.

Examples include:
  • User needs to see a handheld tool
  • User needs to see other real people
  • User needs to see objects that are not in mixed reality but overlaid atop the virtual

 

What isn’t chroma keying good for?

The nature of this technology means that the more immersive your experience is, the better it can leverage chroma keying.  Additionally, it requires a dedicated physical space that you can modify.  Experiences that require the user to be able to see their real environment, or experiences that won’t always be in the same physical location are therefore not well-suited to using chroma keying.

Examples include:
  • User needs to sit in a vehicle in a real room
  • User needs to grab and manipulate virtual objects

 

What content works best with chroma?

The best chroma content will be immersive, single-user, non-interactive environments. This means that experiences like architectural visualizations, virtual tourism, and tool- or weapons-training will shine in a chroma space.

Tips and Techniques for Building Your Own Chroma Space

Planning to build your own chroma space? Next, let’s explore some of the essentials of building your own chroma keying studio with tips and tricks ranging from the materials you'll need to the techniques used.

XR Chroma Wall

How do you build a chroma space?

Creating a chroma space can be as simple as hanging chroma curtains or fabric where you want to have the effect. If you are able to paint, however, using specially-made chroma keying paint on the walls (and floor and ceiling, if you’re ambitious!) of your space will give the best results. At Varjo’s North America HQ we have used Pro Cyc’s Chroma Key Paint, which gives very even and low-reflection coverage.

Compared to using fabric greenscreens in the past, the difference in consistency and quality is significant. By selecting the chroma key paint method, we ensured a non-reflective and evenly surfaced green screen, allowing us to focus on our projects without worrying about changes in hue or tone. Investing in a physical studio space will also save you from having to make adjustments in the future.

Make sure to be extra careful when painting your space however: rushing the process or neglecting protective covers can result in small chromakeyed spots on the floor and shoes, which will be noticeable through the lens.

Learn from Pro Cyc’s blog how Varjo’s virtual production studio was brought to life

What development work is required to use chroma?

None! (unless you want to)

CLICK TO LEARN WHAT SETTINGS TO USE IN VARJO BASE COLLAPSE
Simply enable “Chroma key” in Varjo Base or Varjo Lab Tools, and use the color picker to select your chroma hue.
You can then use the sliders to fine-tune tolerance settings to perfectly suit your chroma space.
We also recommend that you disable “Automatic” mode for “White Balance”, “Exposure”, and “ISO” under “Mixed Reality” settings in Varjo Base/Lab Tools, and instead select fixed values. This will improve the consistency and stability of the chroma blending. You can also enable chroma keying in your software solution, which will allow you to be more selective about which elements of your virtual scene use chroma keying, and which do not.

 

What hardware do you need for a chroma space?

Just a Varjo headset (and tracking solution). That’s it!

 

Can I use chroma keying with [other Varjo Mixed Reality feature]?

Yes! MaskingLiDAR depth occlusion, and Ultraleap hand tracking can all be enabled in tandem with chroma keying.  Depending on the feature in question, you will either need to enable those in your software experience or use Varjo Lab Tools to enable these alongside chroma keying.

  • Lab Tools Masking: Yes!
  • LiDAR hand tracking: Yes!
  • Eye-tracking: Yes!
  • Others: Yes!

Chroma keying can be an immensely powerful technique for levelling-up mixed reality experiences in specific use cases. With the right tools and techniques, you can create a space that seamlessly blends your virtual and real worlds together at the ultimate level.

Want to learn more?

Watch our Chroma Key Webinar on-demand    Talk to sales