NIST Launches Innovation Contests in IoT, Augmented Reality to Boost Emergency Response

The National Institute of Standards and Technology will soon kick off two national innovation contests that aim to strategically leverage augmented reality and internet of things technologies to transform public safety officials’ abilities to respond to emergencies. 

While the IoT contest focuses on the production of smart city data streams to simulate disaster scenarios, its AR counterpart seeks to create AR interfaces for first responders. Each contest envelops research and development phases leading up to a final fourth phase of competition—NIST’s CHARIoT Challenge—which will ultimately integrate the IoT data streams into AR headsets to demonstrate how the wearable and sensor technologies can ultimately help public safety officials to make quicker, more informed decisions.

“We're trying to get people to think innovatively and out of box,” NIST’s Public Safety Communications Research Division Chief Dereck Orr recently told Nextgov about the effort.

In detailing the inspiration behind the new challenge, which offers up more than $1 million in cash prizes, Orr explained that the devices first responders can access to support their work are not far off from the commercial devices people use daily. This means that interactions with those devices require a great deal of hands-on swiping, tapping and pinching.

“And that is not the way we envision public safety interacting with the network in the future,” Orr explained. “You don't give a firefighter or a cellphone that has maps of the building and expect them to walk into a building with their cellphone in their hand—they can't see it in front of their face and they can't touch it with a glove. A SWAT team member can be interacting with their cellphone either. They have their hands preoccupied with a rifle.” 

With that in mind, one of the key areas of research at play is to how to pull all the data and information that first responders have available to them and capture and present it to them in a way that's actionable, and boosts their ability to respond. Researchers on the AR path will explore how holographic and AR information will be used and presented in public safety in the future, both to personnel on the ground and those behind the scenes directing them. This could mean anything from seeing smoke through walls or other critical information not at their fingertips, but right before their eyes. 

IoT comes in because a great deal of that information will likely be streamed through sensor networks, so researchers must ensure that the information is streamlined and doesn’t result in a sort of cognitive overload for the users leveraging them. To help visualize the potential final products, Orr likened them to next-generation heads-up displays, similar to the way, in recent movies, Iron Man’s suit displays much-needed information during the crises he faces. So contestants will ultimately consider questions around what sensors would be most useful to provide relevant information first responders need, as well as the best ways to present that information so that it all enhances operations. 

“And we're not assuming that there is any one way to do that,” Orr noted. “Therefore what we get out of prize challenges like this is that we might see dozens of prototype applications that we can look at and see the benefits, pros and cons for each.”

Deadlines to participate in either contest are May 6. Right now, the program team is working to recruit contestants, which could include technologists, AR developers, gamers, UI/UX specialists, students, smart city and IoT data experts, and beyond. Sarah Hughes, a NIST prize challenge specialist within Orr’s division also shared a glimpse into how the contests and challenges will play out. After submitting their proposals in phase 1, contestants early on will gather feedback on their ideas from the experts involved and initiate the designs and prototyping of either their data streams or their augmented reality interfaces.

“We have great challenge partners that are kind of experts currently in the field, for both augmented reality and IoT visualization and training for public safety parameters and so they'll be available to really help guide, inform and collaborate with the contestants,” Hughes explained. The CHARIoT Challenge partners presently include Magic Leap, which will provide training, as well as actual AR headsets and assets to relevant participants. First Responder Network Authority and FirstNet built with AT&T along with others such as personal protective equipment developer MSA Safety will also provide expertise and insights along the way to help bring the concepts to a rational reality, which Orr noted is like “gold” for contestants. 

“These partners are really vital to our success,” he said.

Following the feedback phase, Hughes said contestants will be expected to demonstrate their proofs as more refined prototypes. The judges will evaluate the creations to move on to the full and final prize challenge competition. The culminating CHARIoT event will bring together actual first responders and others to dawn the newly developed prototypes and assess how they work in real-life simulations. The interfaces created by AR contestants with heads up displays and holograms will take advantage of the data streams provided by the IoT contestants. The judges will use both integrated together to try to complete first responder tasks in emergency scenarios. 

“It should be a great opportunity to bring together public safety innovators, communication technology experts, and really kind of test and advance the state of both the IoT data streams and augmented reality—all at that live event,” Hughes said.

Last year, NIST led a similar prize challenge on haptic interfaces for public safety officials. Just as first responders will test the final prototypes in this year’s contests, Orr noted that in last year’s competition the four final teams were able to test their technologies out with officials at a local fire training center. Judges last year included one retired and several active firefighters, who tried out the developed virtual reality tech. Orr said that often, officials have a bit of skepticism trying out new tools, which is understandable as they frequently work in conditions where their lives could be on the line.

But after the test last year, “they all walked out with an excitement about how this type of technology is not going to take the place of their current procedures, but it can augment their procedures and allow them to do their jobs better. Everyone was surprised and excited about where this could go,” Orr said. In that light, he added that one of the team’s goals this year is “to get public safety excited about this—to show them how this might apply to their future capabilities, so that they want to see this type of thing be developed for them.”

Beyond that, Orr and his team also hope to recruit new innovators into the field and help them realize that their IoT or AR expertise applies to these emergency-driven use cases. They also want the contests to help pave the way for a handful of prototypes that could be used to spur ongoing research and potential product development—“so that in the near future, you could potentially envision seeing some of these technologies put in place,” Orr said.