Unlocking Mixed Reality with Chroma Key: Expert Tips for Creating Your Chroma Studio
Did you know that you can leverage real-time green screening in your immersive applications? Want to achieve a seamless blend of virtual and real elements without coding skills? Read on to learn how to unlock the full potential of mixed reality by utilizing chroma key with Varjo’s mixed reality technology.
Contents:
Chroma Key Pros & Cons: When is the Green Screen the Best Choice Tips and Techniques for Building Your Own Chroma Space
Chroma keying, also known as “green screening” or “blue screening,” is a technique used in video production to superimpose one image or video over another by removing a specific color, usually green or blue, from the background of the top layer. This allows the background to be replaced with a different image or video, creating the illusion that the subject in the top layer is in a different location or environment. With this technique widely-used in the broadcasting and film industry, Varjo revolutionized the game by being the first to offer real-time chroma keying for mixed reality devices.
So what are the benefits and essentials for using this technique in immersive experiences? In this Q&A, we will explore the steps and tools necessary for building your own chroma space, and how you can make the most of this powerful technology to enhance your XR experiences.
Pros and Cons of Chroma Keying: When is the Green Screen Your Best Choice?
Varjo headsets offer built-in chromakey capabilities, making it possible to use chromakeying with any software run through the headset. However, there are specific kinds of experiences that work particularly well with this setup, creating a feeling that you’re stepping into a completely other reality.
Why should I use chroma keying?
Chroma keying enables very high precision occlusion of virtual objects by real objects. Therefore, solutions where the most important feature is the ability of the user to accurately see real objects best take advantage of chroma keyed environments. A “real object” could be something handheld, furniture or even another person in the room — everything real will be visible “on top of” the virtual elements. As we’ll discuss further down, you can also combine chroma keying with other Varjo features to ensure that you are still able to have physical and virtual elements interact appropriately.
Examples include:
- User needs to see a handheld tool
- User needs to see other real people
- User needs to see objects that are not in mixed reality but overlaid atop the virtual
What isn’t chroma keying good for?
The nature of this technology means that the more immersive your experience is, the better it can leverage chroma keying. Additionally, it requires a dedicated physical space that you can modify. Experiences that require the user to be able to see their real environment, or experiences that won’t always be in the same physical location are therefore not well-suited to using chroma keying.
Examples include:
- User needs to sit in a vehicle in a real room
- User needs to grab and manipulate virtual objects
What content works best with chroma?
The best chroma content will be immersive, single-user, non-interactive environments. This means that experiences like architectural visualizations, virtual tourism, and tool- or weapons-training will shine in a chroma space.
Tips and Techniques for Building Your Own Chroma Space
Planning to build your own chroma space? Next, let’s explore some of the essentials of building your own chroma keying studio with tips and tricks ranging from the materials you'll need to the techniques used.
How do you build a chroma space?
Creating a chroma space can be as simple as hanging chroma curtains or fabric where you want to have the effect. If you are able to paint, however, using specially-made chroma keying paint on the walls (and floor and ceiling, if you’re ambitious!) of your space will give the best results. At Varjo’s North America HQ we have used Pro Cyc’s Chroma Key Paint, which gives very even and low-reflection coverage.
Compared to using fabric greenscreens in the past, the difference in consistency and quality is significant. By selecting the chroma key paint method, we ensured a non-reflective and evenly surfaced green screen, allowing us to focus on our projects without worrying about changes in hue or tone. Investing in a physical studio space will also save you from having to make adjustments in the future.
Make sure to be extra careful when painting your space however: rushing the process or neglecting protective covers can result in small chromakeyed spots on the floor and shoes, which will be noticeable through the lens.
Learn from Pro Cyc’s blog how Varjo’s virtual production studio was brought to life
What development work is required to use chroma?
None! (unless you want to)
What hardware do you need for a chroma space?
Just a Varjo headset (and tracking solution). That’s it!
Can I use chroma keying with [other Varjo Mixed Reality feature]?
Yes! Masking, LiDAR depth occlusion, and Ultraleap hand tracking can all be enabled in tandem with chroma keying. Depending on the feature in question, you will either need to enable those in your software experience or use Varjo Lab Tools to enable these alongside chroma keying.
- Lab Tools Masking: Yes!
- LiDAR hand tracking: Yes!
- Eye-tracking: Yes!
- Others: Yes!
Chroma keying can be an immensely powerful technique for levelling-up mixed reality experiences in specific use cases. With the right tools and techniques, you can create a space that seamlessly blends your virtual and real worlds together at the ultimate level.