Bringing a Human-Centered Approach to Immersive Experiences 

Imagine a world where instead of pulling up Google Maps on your phone, you can visualize turn-by-turn directions in 3D from your contact lenses. Now imagine the same technology being employed to train the next generation of fighter pilots and Navy SEALs. Imagine making decisions based on information presented in the way we spent our lifetime perceiving in 3D.  That’s where extended reality, or XR, innovation is heading, and the impact on the Department of Defense and federal/civilian agencies' practices will be transformational. 

Extended reality is a broad term that encompasses immersive technologies across virtual reality, augmented reality and mixed reality. This approach extends reality by simulating or adding to real-world scenarios via digital tools. It’s revolutionary because of how people perceive. 

Traditionally people interface with information primarily in two dimensions—think classroom whiteboards, PowerPoint presentations and online video tutorials. XR technology is human-centered because it delivers information the way humans have always perceived the world throughout our lives: in 3D, not via spreadsheets and 2D pictures. XR provides users with the ability to see the problem from the same perspective through which they experience real life, enabling faster, more effective decision-making. Using headsets, eyewear, mobile devices and software, XR empowers people with a hands-on virtual experience to accelerate retention of skills and demonstrate capability faster than any other tool. Within three years, XR will be so ubiquitous that nearly everyone will have access to some sort of XR technology impacting their daily lives. But to realize the full value of XR, we have to leverage emerging technology to open up new opportunities and break down barriers to adoption. 

A Perfect Match: XR + AI

One of the biggest advantages and opportunities XR delivers over traditional simulation technology is its ability to integrate with data analysis tools across artificial intelligence, machine learning and sophisticated analytics. Imagine a warfighter moving through a scenario where not only do they have a realistic 3D view of the battlespace, but are also viewing real-time intelligence data on adversary positioning and resource allocation through a single pane of glass. As they move through the XR environment, AI and ML tools dynamically update ever-changing data to inform decision-making. This is the future of multi-domain warfare, and it’s critical for personnel to prepare for decision support in these environments. 

The other benefit is AI-directed learning. AI and ML are capable of providing different cues and scenarios based on user decisions. Personnel could go through XR modules that adapt to changing conditions depending on a host of variables, measuring effectiveness along the way while mitigating the risk of falling into negative learning experiences. 

Overcoming the Barriers to Adoption: Mobility, Culture & Cost 

Like any emerging technology, adoption requires proven value, ease-of-use and accessibility. 

XR is still often viewed as a “novelty” technology within the DOD, and it will have to win over leadership in three primary areas to realize its full potential to transform human performance. 

Far from the enormous, expensive simulation environments that the DOD has used for decades—think flight simulators—one of the huge benefits of XR software is that it’s flexible enough to be placed on a variety of form factors. But skeptics are still concerned that the fit, form and bulk of some of the devices, from headsets to eyewear, can hinder or physically limit users. The good news is that as technology progresses, mobility and ease-of-use are dramatically improving. There is a not-so-distant future where personnel each have XR glasses and contact lenses enabled by AI and ML, guiding them through immersive training scenarios. And existing headsets are getting lighter, more comfortable, more wireless and more mobile every day.  Also, we should not limit XR platforms to just head-mounted displays; mobile smartphones with XR platforms are contributing to the ubiquity of XR, and we are just on the cusp of harnessing the potential.

Change is always challenging, especially when a new technology is fighting the “novelty” perception mentioned previously. The best way to prove the value of the technology is to show what it’s already done. The Air Force’s Pilot Training Next program has utilized augmented reality and virtual reality systems to significantly optimize overall pilot training time while improving retention. Using a digital classroom in which low-cost simulators replaced books, personnel learned at their own pace and embraced gamification techniques to optimize competency. It’s the type of success story that can go a long way toward educating leaders on the benefits of XR-powered training across speed and performance. 

Lastly, industry has to find innovative ways to streamline costs and make the technology accessible to every DOD service member that needs it. XR has the reputation for being expensive, which is sometimes true, but price points are coming down quickly. XR is often much less expensive than sending personnel into the field for training for example, and industry continues to drive efficiencies across form factors and integrators provide risk mitigation strategies which aid in investment obsolescence in this exponentially improving environment. Even so, there is more work to be done. 

We’re on the cusp of a sea change in immersive experiences aided by extended realities. Frost and Sullivan predicted that XR based technologies will be as ubiquitous as laptops by 2030, and pockets of the DOD are already leading the charge and realizing transformational mission benefits in retention, speed and performance. Industry has a responsibility to innovate and drive increased flexibility and accessibility to pave the way for a new world of increased performance aided by immersive experiences.  

Bob Kleinhample is vice president of immersive technologies at SAIC.