Video Pass-Through XR Changes Reality As You Know it
Mixed reality is set to become the most revolutionary technology since the invention of the graphical user interface.
But for mixed reality to be valuable, it’s got to be convincing – blending real and virtual to the point that it’s impossible to tell where reality ends and the virtual world begins. XR-1 Developer Edition is the only device that’s ever been able to do that.
Mixed reality is all about merging virtual content with the real world. To do that, today’s augmented reality headsets use optical see-though to create holographic images that float in front of you. The problem is, those images are hazy and ghostlike, because optical see-through devices can only display light, not black or opaque content.
Plus, those devices also have to make big compromises on field of view, resolution, or both.
So far augmented or mixed reality hasn’t even come close to “hard AR” – the holy grail academic term used to describe a reality where you can no longer tell what’s digital and what’s real.
Varjo XR-1 Developer Edition changes that.
By using video pass-through instead of optical see-through, XR-1 Developer Edition completely and convincingly merges real and virtual, making it the only device to achieve photorealism in mixed reality.
See the difference between an optical see-through mixed reality device and Varjo's video pass-through mixed reality.
OPTICAL SEE-THROUGH DEVICE
The power of pass-through
XR-1’s video pass-through technology allows you to do things that are completely impossible with optical see-through systems like HoloLens or Magic Leap. Virtual objects can be black or opaque and appear as solid as anything in the real world. Colors are perfectly rendered and appear just as they should. You can also add, omit and adjust colors, shadows and light in the virtual world and the real world – something that’s never been possible before. And real-time camera data is used to calculate global illumination models that allow shadows and light to naturally interact with virtual objects in real time – just as they would with real-world objects. Virtual objects can even illuminate, cast shadows over, or reflect the world around them. They can also appear as transparent and refract the light from the real world behind them.
Importantly, XR-1 Developer Edition also allows you to see your own body as you interact with virtual objects, the real world around you, and even your colleagues as you collaborate on mixed reality projects. And when you want to switch back to “real” reality, it only takes a click of a button.
So how does Varjo’s pass-through technology work?
Video pass-through uses cameras to digitize the world in real time. That data is then combined with virtual content inside our GPU and shown to users through our device – completely blurring the lines between real and virtual. Varjo is the only company on the planet with a resolution high enough and a latency (< 20 ms) low enough to pull this off.
Video pass-through follows the same general logic as our Bionic Display™. Visual quality and bandwidth are optimized by sampling video streams from each camera with different resolutions and using eye tracking to determine which area of the sensor should be sampled in the highest resolution. So the highest resolution will always be wherever the user is looking at any given time.
Just as the human eye does, XR-1 Developer Edition then samples the surrounding areas in lower resolution to make sure the transfer speed (below 10Gb/s) can be handled by modern computers. To put this into context, the human eye sends peripheral images to the brain at just 10 Mb/s.
All this means that XR-1 Developer Edition runs at around 35 times the resolution of HTC Vive Pro mixed reality, for example, and each individual pixel is of a much higher quality.
None of this would be possible without our close collaboration with our amazing partners, Sony and Socionext. XR-1 Developer uses Sony cameras with an IMX 274 sensor. The two cameras’ ultra-high resolution totals a whopping 24 million pixels at 90 frames a second. And the Image Signal Processors (ISPs) made by Socionext are powerful enough to process 2.3 billion pixels per second.
Revolutionizing work in a range of fields
We introduced the XR-1 Developer Edition to the world for the first time at Augmented World Expo 2019, and with that we gave a glimpse into the magic that can be made with video pass-through.
We collaborated with Volvo Cars to create a truly ground-breaking demo of a real car being safely driven while wearing XR-1 Developer Edition and interacting with totally convincing virtual elements – including a virtual moose that the driver needs to avoid. Today, Volvo is using XR-1 Developer Edition to design and develop the world’s safest cars. With Varjo’s photorealistic mixed reality and eye tracking, Volvo is now able to do design and safety studies on cars that haven’t even been built yet.
We have also collaborated with Bohemia Interactive Simulations to create a hybrid cockpit simulator where the world outside the cockpit is illustrated with VR and the cockpit is real and enhanced with video see-through. This demo was showcased at I/ITSEC 2019.
Professionals in engineering, design, simulation, and research can use XR-1 Developer Edition to develop and interact with photorealistic 3D models while collaborating with others in real life and real time. And our integrated eye tracking provides unprecedented and deeply reliable insight into how people experience and interact with a products, scenarios and environments.
Now available and shipping
XR-1 Developer Edition marks the start of a totally new era of professional immersive computing – where virtual and real realities truly become one. XR-1 Developer Edition is now available and shipping, and we can’t wait to see our enterprise partners explore the technology and develop entirely new realities of their own.