Case Explico: How Mixed Reality Helps Forensic Engineers Educate Jurors in the Courtroom
Explico is developing mixed reality solutions to help prove complex issues to juries in the courtroom. Using the Varjo XR-3 headsets, the company can present fully-immersive and accurate reconstructions of various scenarios, demonstrating what happened and helping to reach a resolution.
Unraveling the details of what happened
Did the specific lighting conditions and nearby structures affect a driver’s ability to perceive the distance of an oncoming motorcycle or passing pedestrian? Was a downed powerline open and obvious to a nearby individual walking by? Did a carpeting’s pattern create an optical illusion thereby obscuring a step, which ultimately caused an individual to fall and become injured? Legal matters involving accidents, failures and even serious crimes often hinge on whether one side can effectively communicate critical, complex issues to the jury.
Answering these questions objectively is not easy. To help prove what happened and effectively communicate it to a jury, 3D animations have long since become an important tool in courtrooms because they can convey very complex issues in a compelling and understandable way. Example use cases include showing a scene of a crime, illustrating the inner workings of a failed mechanical device, and demonstrating the events that lead to an injury.
In these cases, accuracy is paramount. Accurate, high-fidelity representations of objects and environments can be generated from 3D laser scanning (LiDAR) devices. These 3D reconstructions and animations are more informative than traditional photographs or video, but they still have limitations.
The biggest challenge is that 3D content is still being viewed on traditional 2D screens. This results in a lack of immersion and interactivity. Even the most detailed 3D animation, when viewed on a conventional display, will not be able to provide the sense of space and depth that is available in XR. Consequently, for certain accidents and issues, XR’s ability to provide an immediate understanding of what a person may have observed from their perspective, could provide critical understanding to jurors when reaching a verdict.
Things are about to change, though. Immersive technology is already revolutionizing how 3D scenes and animations are used in mediations, and it won’t be long before virtual and mixed reality headsets make their way to the courtroom. And at the heart of this development are Explico’s Steve Rundell, PhD, PE and Steve Arndt, PhD, CHFP.
The challenge: Using an immersive virtual experience for more accurate depictions of events
One of Explico’s unique areas of expertise is using 3D engines such as Unity and Unreal Engine to create amazing, highly accurate interactive 3D models and animations that show what happened. This has led them to work on numerous high-profile cases where precisely proving the course of events, and being able to convey that understanding to a jury, is essential.
Explico’s goal has always been to make the experience even better by leveraging the latest technologies. To overcome the limitations of 2D screens, Explico decided to explore immersive technologies as a potential solution.
The reason they chose mixed reality over virtual reality was simple: the possibilities of this technology are far more flexible, and the experience is far more intuitive to non-technical users and less jarring to people not accustomed to virtual reality, which includes most judges and jurors.
The solution: Varjo XR-3 mixed reality
While Explico had tried other headsets before, Varjo’s industry-leading resolution and ease of use proved to be a great match for their needs.
Varjo XR-3 has allowed them to present fully-immersive and accurate reconstructions of various scenarios, clearly demonstrating what happened or was observable, and helping reach a resolution such as a settlement. Thanks to hand tracking and automatic IPD adjustment, the headset is also easily usable by any kind of user, no matter what their proficiency level with immersive technologies might be.
Varjo XR-3 has been vital for Explico’s work. The visual quality and the seamless blend of the real world and the virtual world ensures that when someone is wearing a headset, real-world conditions can be presented accurately. Visual acuity needs to be close enough to real life for the technology to bring value, and XR-3 achieves this in an unmatched way. For example, it is possible to show what an individual would actually be presented with since objects look natural and unpixelated – something no other headset can realistically achieve.
Visualizing Scenes in a User-Friendly Way
Mixed reality also makes the experience less jarring than being immersed in a fully virtual scene right from the start. This is especially useful with people who might not even have heard of virtual reality before and do not necessarily know what to expect when they are put in such a scene.
Ease of use is extremely important once headsets are used in legal proceedings. Jurors, judges and lawyers cannot be expected to be proficient in working with headsets. The technology needs to be intuitive to any kind of user. Once this technology is deployed in jury trials, there simply won’t be time to teach each and every juror to operate controllers, navigate complex settings, or adjust interpupillary distances so they don’t get nauseous.
This is why accurate and natural hand tracking is a must have: there is no need for learning any kind of controller once a person can simply use their own hands to interact with an environment. Automatic IPD removes the need for manual visual calibration so the device can simply be put on and adjusted with the click of a single button, without any expertise needed from the user. This ensures that there is no eye strain, everything can be seen clearly, and there is no cyber sickness.
Another major benefit of this technology is that it can show events in real-life scale inside the courtroom, something that would be impossible with physical objects and places alone. For example, it is simply not realistic to bring large machines or vehicles into a courtroom, or expect a jury or judge to travel to the actual accident site in every single case.
The technology was also relatively easy to take into use as the Explico team was already using Unity to create 3D animations. Converting them for use with an XR headset was simply a natural step forward.
The result: Immersive scenes will be the future of courtroom animations
While not deployed during a jury trial within the United States yet, these immersive solutions have already been used as part of arbitration and mediation proceedings that ended up in a settlement outside the courtroom. Judges and arbitrators have already been able to view scenarios in mixed reality. It’s only a matter of time until jurors do so as well.
However, given how compelling many of these interactive demonstrations are using the Varjo XR-3, Steve Rundell of Explico anticipates that most cases will settle prior to trial. Specifically, once one side has had an opportunity to experience the other side’s compelling and interactive VR experience, they won’t want to run the risk of having a jury be exposed to the same evidence. Consequently, many lawyers will fight to keep VR out of the courtroom if it doesn’t aid in their case. Fortunately, a combination of Varjo’s incredibly high visual acuity combined with Explico’s previous and future peer-reviewed research will make exclusion of this evidence difficult for attorneys.
Explico’s Steve Rundell sees that it is only a matter of time before the technology is deployed during a jury trial as well, and thanks to Varjo XR-3, Explico is ready to be a part of that monumental shift. He believes the first cases where the technology is going to be used will most likely end up in court fairly soon – perhaps already this year.
“The technology has reached a sufficient level of maturity to become widely adopted. It has the chance to revolutionize court proceedings, especially in tricky cases where you need to depict something that can only be depicted in XR. For example, nobody wanted to touch animations in the 80’s, but they are now a staple in many court cases. I predict a similar shift towards extended reality will happen eventually, because the degree of accuracy and how well it can reflect the real world is simply unmatched, and Varjo XR-3 is a perfect device for making that happen,” Rundell says.
Learn more about other pioneering use cases for VR/XR:
How Air Zermatt is Setting New Standards for Helicopter Pilot Training with Varjo
May 26, 2023
Lockheed Martin and Varjo Demonstrate Next-Gen Immersive Cloud Streaming Technology for Training
May 17, 2023