To manage big data, go beyond IT

If you have a big data program to manage, you need a talented IT person, right?

Not necessarily. Big data brings its own challenges and calls for some very specific skills, according to members of a panel discussion at a Dec. 3 hosted by Nextgov.

Although data has always been around, the amount seen today is what makes management so challenging, said Micheline Casey, principal at CDO, LLC and former chief data officer for the State of Colorado.

“The other thing that’s critical,” she said, “is that we have this nexus of what’s happening with the interconnected systems. The dynamic pace of change of innovation that’s just really requiring people to think proactively and not just reactively about data and how manage data assets.”

Big data is just not about the volume of information, said Jeff Butler, director of research databases at IRS Research, Analysis and Statistics, but the many variations of it that add to the challenge. These new types of data, whether unstructured or textual, are forcing a reevaluation of skills in analyzing those variations, he said.

Also, Butler said, organizations traditionally used to manage data in monthly, quarterly or annual cycles. Today, that tends to be done in real-time, or close to it. The question then for agencies on the federal level, he said, becomes how to adapt to those real-time analytics models?

Today “data just happens” and is generated without much thought, said Michael Rappa, director of the Institute for Advanced Analytics and professor at North Carolina State University. Thirty years ago, organizations collected data with purpose and intent. The process required time, money and energy that needn’t be spent if there was no identified need for the data, he said.

Today, Rappa said, the challenges for those who manage and analyze all this data lies in how to do it in a predictive way and very quickly. “That’s the talent pool that’s not really in place,” he said. “That’s the career track that really doesn’t exist in most agencies just quite yet .”

The right kind of person to take on the challenge is often not an IT person, Rappa stressed. Agencies are now starting to work on figuring out what the operational role looks like, and the “data scientist” job title gets used a lot. However, the term is misleading because the person who handles the big data challenge “is not a scientist and they shouldn’t be totally fixated on data,” Rappa said.

Agencies need to be more data-centric and think of data upfront, not just as an afterthought, Casey said.

“Building a culture around data is very different from the culture virtually all state and federal agencies and commercial sector business have today, quite frankly,” she said. “It doesn’t matter if you get the best data scientists in a room; if no one across the organization knows what to do with the data, the insight they come up with doesn’t matter. Data is a business issue, not an IT issue.”

Agencies also need to create an organizational capacity to take in, consume and distribute data across their ecosystem to make use of the insights data scientists provide, Casey suggested.

“Otherwise it’s not doing anyone any good -- having a room full of data scientists and Hadoop clusters,” she said.