Analytics guru Nate Silver offers advice for agencies

There is no shortage of hype about big data in the federal IT space, and separating the buzz from tangible benefits has thus far proven difficult.

So says Nate Silver, who recently authored a book about predictive analytics and successfully predicted the presidential election outcomes in all 50 states in his Five Thirty Eight blog for the New York Times.

Silver, speaking Feb. 12 at the Adobe Government Assembly in Washington, D.C., cautioned the audience that big data is "not a panacea" and won't single-handedly "solve all our problems."

Big data may well become one of the most potent tools in the government's IT arsenal and lead to countless data-driven solutions in the near future, but Silver stressed that it will still take good, old-fashioned brain power to engineer and makes sense of them.

"The people in this room are still pretty important," Silver said. "Don't become too enamored with computers."

In fact, he predicted, imaginative individuals will play a critical role in determining the future of big data in the federal sector.

Silver recounted several "cautionary tales" in big data drawn from his book, "The Signal and the Noise: Why so Many Predictions Fail, But Some Don't." And he offered three suggestions to federal agencies looking to thrive in the era of big data:

Think probabilistically, embrace uncertainty

Silver, who once earned six figures as a poker player, used an analogy from his favorite game to describe how agencies can employ big data. Playing the percentages, he said, can be beneficial, and routinely making the right play generally leads to foreseeable outcomes. But embracing uncertainty -- throwing a change-up now or then -- can lead a player to unexpected profit on the poker table, and could lead agencies to unexpected successes that boost their cost-benefit margins.

Sweat the small stuff when you face big problems

As a tool, big data can unlock all kinds of insights from massive amounts of data, but small changes in the way we approach humongous data sets can drastically change outcomes. Seemingly minor details should never be overlooked, Silver said.

Question your assumptions; question everyone else's assumptions, too

Making sense of the noise can be challenging, but finding signals in the noise does not always mean they are connected, Silver said. It's also important to be cognizant of what people are saying, and why.

"Beware of people who tell you (big data) will solve all your problems," Silver said.

Big data is a challenging concept for any user, but Silver noted one more challenge facing the public sector that perked the ears of the agency-heavy audience.

"Bureaucracy is the enemy of imagination," he said, as the crowd erupted into laughs, smiles and knowing glances across the table.