Road to I/ITSEC 2022: Co-Pilot Training in Mixed Reality by SAIC
To celebrate I/ITSEC 2022 taking place in a few weeks, Varjo is highlighting innovative partners from last year’s event.
Using the Varjo headset, SAIC allows pilots to see their own hands, interact with co-pilots, and touch physical cockpit controls; all while immersed in virtual airspace.
Finding added value in mixed reality
Science Applications International Corporation (SAIC) is an American company developing groundbreaking simulation and technology solutions. Learn how SAIC leverages the Varjo headset in their simulation solutions, including a mixed reality simulator designed for training two pilots to operate an aircraft as a team.
Watch the interview with SAIC from I/ITSEC 2021
Challenge: How to train pilots as a team in a virtual environment
By its nature, regular virtual reality training is a very secluded and individual experience, so one of the challenges is how do you train pilots in a virtual cockpit when they need to be sitting side-by-side and working together in real life aircraft. While you could try to enable collaboration with virtual avatars in VR or with real people in augmented reality, both of these technologies have some shortcomings for achieving realistic two-person training.
In augmented reality, the field of view is much narrower, which makes it impossible for pilots to accurately train the use of peripheral vision. The AR image is also ghostlike and non-immersive. Virtual reality allows for complete immersion, but when you need to train two pilots to operate together as a team, and they need to be able to communicate both verbally and non-verbally in the cockpit, the technology simply isn’t compatible. VR is also less optimized for operating physical controls.
These incompatibilities are exactly why a mixed reality solution is needed.
“I’m a pilot with about 25 years of experience. And the first time I put on a Varjo headset, what I saw was a level of realism that I hadn’t seen in my pilot training career, and in my career at SAIC.”
Marshall Groves - Senior Program Manager at SAIC
Solution: Industry-leading mixed reality enables truly training as a team
Thanks to the Varjo mixed reality headset, SAIC’s pilot training solution allows two pilots to seamlessly train in the same virtual scenario, while also seeing and hearing each other. The mixed reality experience utilizes Varjo’s industry-leading video pass-through to seamlessly blend together real-world objects (such as aircraft controls and the co-pilot next to you) with virtual content such as the simulated flight scenario.
This allows the trainees to effortlessly interact with both digital and real-world objects, resulting in increased immersion and training results. Mixed reality allows natural communication and interaction, so the pilots can train as a real team inside the virtual scenario, not as two, disconnected individuals. This helps them develop and practice all the skills that would be necessary when operating a plane with a pilot and co-pilot.
