Report: Dark Data Plagues Federal Organizations

While government leaders across the globe are excited about the unleashing artificial intelligence in their organizations, most are struggling with deploying it for their missions because they can’t wrangle their data, a new study suggests.

In a survey released this week, Splunk and TRUE Global Intelligence polled 1,365 global business managers and IT leaders across seven countries. The research indicates that the majority of organizations’ data is “dark,” or unquantified, untapped and usually generated by systems, devices or interactions.

AI runs on data and yet few organizations seem to be able to tap into its value—or even find it.

“Neglected by business and IT managers, dark data is an underused asset that demands a more sophisticated approach to how organizations collect, manage and analyze information,” the report said. “Yet respondents also voiced hesitance about diving in.”

A third of respondents said more than 75% of their organizations’ data is dark and only one in every nine people reports that less than a quarter of their organizations’ data is dark.

Many of the global respondents said a lack of interest from their leadership makes it hard to recover dark data. Another 60% also said more than half of their organizations’ data is not captured and “much of it is not even understood to exist.”

Research also suggests that while almost 100% of respondents believe data skills are critical for jobs in the future, more than half feel too old to learn new skills and 69% are content to keep doing what they are doing, even if it means they won’t be promoted.

“Many say they’d be content to let others take the lead, even at the expense of their own career progress,” the report said.

More than half of the respondents said they don’t understand AI well, as it's still in its early stages, and 39% said their colleagues and industry don’t get it either. They said few organizations are deploying the new tech right now, but the majority of respondents do see its potential.

The report also suggests that one of AI’s central values is its ability to work with vast quantities of data at remarkably high speeds. With more information to analyze, it can provide deeper and more accurate insights. Researchers said that many of these organizations’ dark data could provide an enormous untapped resource of information for AI to analyze. They recommend that AI-powered analytics tools could help make the untapped data ready for analysis on a scale that seems impossible with today’s methods.

“Today’s dark data could one day be an accelerant for even greater AI performance. Thus, the advent of AI and the value of dark data go hand in hand,” the report said.