Video Pass-Through XR Changes Reality As You Know it

Mixed reality is all about merging virtual content with the real world. To do that, today’s augmented reality headsets use optical see-though to create holographic images that float in front of you. The problem is, those images are hazy and ghostlike, because optical see-through devices can only display light, not black or opaque content.

Plus, those devices also have to make big compromises on field of view, resolution, or both.

So far augmented or mixed reality hasn’t even come close to “hard AR” – the holy grail academic term used to describe a reality where you can no longer tell what’s digital and what’s real.

Varjo changes that.

By using video pass-through instead of optical see-through, Varjo’s mixed reality device completely and convincingly merges real and virtual, making it the only device to achieve photorealism in mixed reality.

Varjo’s video pass-through technology allows you to do things that are completely impossible with optical see-through systems like HoloLens or Magic Leap.

Virtual objects can be black or opaque and appear as solid as anything in the real world. Colors are perfectly rendered and appear just as they should. You can also add, omit and adjust colors, shadows and light in the virtual world and the real world – something that’s never been possible before.

And real-time camera data is used to calculate global illumination models that allow shadows and light to naturally interact with virtual objects in real time – just as they would with real-world objects. Virtual objects can even illuminate, cast shadows over, or reflect the world around them. They can also appear as transparent and refract the light from the real world behind them.

Importantly, Varjo also allows you to see your own body as you interact with virtual objects, the real world around you, and even your colleagues as you collaborate on mixed reality projects. And when you want to switch back to “real” reality, it only takes a click of a button.

Video pass-through uses cameras to digitize the world in real time. That data is then combined with virtual content inside our GPU and shown to users through our device – completely blurring the lines between real and virtual. Varjo is the only company on the planet with a resolution high enough and a latency (< 20 ms) low enough to pull this off.

Video pass-through follows the same general logic as our Bionic Display™. Visual quality and bandwidth are optimized by sampling video streams from each camera with different resolutions and using eye tracking to determine which area of the sensor should be sampled in the highest resolution. So the highest resolution will always be wherever the user is looking at any given time.

Just as the human eye does, Varjo then samples the surrounding areas in lower resolution to make sure the transfer speed (below 10Gb/s) can be handled by modern computers. To put this into context, the human eye sends peripheral images to the brain at just 10 Mb/s.

All this means that Varjo’s mixed reality runs at around 35 times the resolution of HTC Vive Pro mixed reality, for example, and each individual pixel is of a much higher quality.

Professionals in engineering, design, simulation, and research can use Varjo’s mixed reality to develop and interact with photorealistic 3D models while collaborating with others in real life and real time. And our integrated eye tracking provides unprecedented and deeply reliable insight into how people experience and interact with a products, scenarios and environments.

We collaborated with Volvo Cars to create a truly ground-breaking demo of a real car being safely driven while wearing a Varjo headset and interacting with totally convincing virtual elements – including a virtual moose that the driver needs to avoid. Today, Volvo is using Varjo headsets to design and develop the world’s safest cars. With Varjo’s photorealistic mixed reality and eye tracking, Volvo is now able to do design and safety studies on cars that haven’t even been built yet.

We have also collaborated with Bohemia Interactive Simulations and many other simulation companies to create a hybrid cockpit simulator where the world outside the cockpit is illustrated with VR and the cockpit is real and enhanced with video pass-through.

Varjo marks the start of a totally new era of professional immersive computing – where virtual and real realities truly become one.

Varjo XR-3 is now available and shipping, and we can’t wait to see our enterprise partners explore the technology and develop entirely new realities of their own.

Get pricing and purchase

Request a demo