Bringing the Virtual World into Reality: Manned Flight Simulators Integrate VR/MR Technology for CMV-22 Platform
For decades, pilots have taken to flight simulators at Naval Air Warfare Center Aircraft Division’s (NAWCAD) Manned Flight Simulator (MFS) to train, test software and equipment, experiment, develop flight envelopes and even investigate aerial accidents in a safe and inexpensive manner. As newer aircraft and technology are created, these simulators need to be upgraded to reflect these developments, as was the case with one of the Navy’s newest aircraft, the unique CMV-22 Osprey.
When exploring options for upgrading existing platforms to meet necessary specifications, the team at NAWCAD learned that they could develop their own VR and mixed reality (MR) motion platform and software for the CMV-22 for a fraction of the existing motion platform upgrade cost— roughly $500,000—and then tailor it to be used with other aircraft as demand arises.
“There are many benefits to moving toward this virtual display,” said Matt Mueller, the Division Head at Manned Flight Simulator. “The biggest benefit being that when you have a facility where you are trying to put more and more simulators in, the advantage of being able to make your footprint smaller is huge. At this facility, we can fit nine simulators in here. But, if we were able to pull out the display systems with projectors and put something like the [VR/MR platform] in, we can have several more motion-capable simulators. They are cheaper to maintain, have a smaller footprint and offer us more capability.”
Inside Manned Flight’s VR/MR Motion Platform
The new VR/MR motion platform sits in the center of what was once a cubicle office area at Manned Flight, a space that was repurposed once the COVID-19 pandemic hit. The platform sits on the floor, with hydraulic pistons extending upward and attaching below the cockpit seat. An array of computer servers and equipment are tucked away in towers to the side of the room, powering the software and platform itself.
To get into the pilot seat, one must climb a short ladder and be limber enough to maneuver into the seated position. Once situated, the pilot can don the VR headset, put their hands on the throttle and flight stick and toggle the foot pedals.
After a brief calibration, the pilot can look around and see the insides of the cockpit, as well as the deck view of an aircraft carrier deployed at sea. Pilots can then practice take off and landing on the carrier, while others can observe what the pilot sees on a computer monitor only a few feet away.
Innovating Training for the Future
The installation of the newest platform began in earnest about 18 months ago, but the push to move toward VR capability began even earlier.
“About four years ago when we were at the tail end of our first virtual reality project, our crew chiefs said they really enjoyed the VR aspect and that’s when we really started working on the mixed reality concept,” said Robert Calvillo, aerospace engineer at NAWCAD.
“We had pilots come in and put on the VR headset and they were essentially able to see outside the windshield of the real cockpit and see the virtual environment, but when they look inside the cockpit, they see the real world, their real hands and instrument panels. Everyone started to see that virtual reality and mixed reality had some really good potential, so we needed to expand on it further.”
Building on the Potential to Revolutionize Simulation
James Pritchard, the Flight Dynamics Rotary Wing Team Lead in the Aeromechanics Division with NAWCAD, first tested out the VR/MR platform in January 2023 and provided his feedback to the team, stating that the visual scene and ship models were rendered very accurately and believable.
“We generated a list of items necessary for further development,” Pritchard said. “However, the overall assessment is very positive. This new technology has the potential to revolutionize modeling and simulation in the shipboard environment.”
The software currently uses the Unity 3D Game Engine, which is also used in the gaming industry. Calvillo said they have another team that is working to develop software using the Unreal Engine, another widely popular 3D game engine used throughout the gaming industry to render games.
The Next Frontier of VR/MR Simulators
Matt Mueller from Manned Flight said the Joint Simulation Environment aims to set up entire facilities comprised of only mixed reality headsets: “no projectors or display systems as we see today.”
“There are problems to solve, which is why we keep tackling them,” Mueller said. “One of the big goals that we are shooting toward is to be able to have a facility of 30-plus simulators all flying in close proximity to each other.
“The advantages and the versatility you get with the VR headsets and the advances that are being made, I expect they will become the more accepted solution moving forward.”
NAWCAD’s Pritchard said he sees the simulators as a cost-saving measure as well.
“Here at Naval Air Systems Command, one of our primary data products comes through flight test, which is resource intensive, both in cost and schedule. This motion platform with MR headset has the potential to augment, and in some cases even replace flight test, thereby realizing a significant cost savings in critical aspect of data collection,” he said.
“Regarding the larger scale naval fleet technology application, this unique simulator has the potential to provide high fidelity simulation for training, certification and currency, which would be a major breakthrough for Naval Aviation simulation.”
This article was originally published in Naval Aviation News, Winter 2023 edition.
Text by Rob Perry and images provided by Todd Frantom.