Demand grows for data scientists

The world is full of data – 2.5 quintillion new bytes of it every day – but despite humanity's growing technological prowess, it still takes well trained, intelligent and intensely curious minds to sift the signal from the noise.

That is why the Harvard Business Review recently dubbed the position of data scientist as the "sexiest job of the 21st century," citing a 15,000 percent increase in job postings from 2011 to 2012 -- growth almost entirely due to organizations battling to keep their heads above the big data tidal wave.

Data science's sex appeal is certainly debatable, but beyond doubt is the demand for their services -- and the data-driven successes in the private sector. The Googles, LinkedIns and Facebooks of the world have all spoken publicly about how these master data manipulators have turned big data into big insights and big profits.

The same incentives for private sector companies to hire data scientists don't always exist in the federal sector, but data scientists are certainly making noise within federal agencies, too. It is part of a "paradigm shift" in data creation, according to Dr. Robert Hummel, vice president and Chief Scientist at the Potomac Institute for Policy Studies.

Hummel, formerly a program manager at the Defense Advanced Research Projects Agency, was part of a panel discussion on the emerging role of data scientists from Carahsoft's third annual Government Big Data forum in Washington, D.C., on April 4.

Hummel explained that, decades ago, agencies spent massive amounts of money collecting information. Collected information was analyzed, of course, by the 1970s and 1980s predecessor to today's data scientists.

"Now the data is flooding us, it flows to us naturally from many communities, and that has allowed for a large range of new possibilities," said Hummel, who said the field is too new to say how many data scientists the government needs.

However, he said, data science -- by teasing out the occasional needle in the proverbial big data haystack -- is already signaling its importance to the federal government.

"I think the government is awakening to the idea that data science can provide models that have great utility for a variety of missions that are important to the government," Hummel said.

And as agencies look to do more with their data, they may find that some of the talent is already there. Data scientists have been around for years, but were usually outside the spotlight and working under different job titles.

"Data scientists are not a new role, it's just a new way of calling themselves," said Joey Echeverria, Principal Solutions Architect at Cloudera.

Right now, it is tough to tell how many data scientists the federal government employs. As Echeverria pointed out, not everyone who does the work of a data scientist is actually called by that title.

A recent poll by the Government Business Council of 313 executives in 27 federal agencies said only four percent of execs were currently hiring data scientists. The seeming lack of hires may be related to sequestration, tight budgets or a lack of big data strategies at the top of organizations. (Read FCW'searlier coverage.)

When agencies do hire data scientists, they should look for a very particular set of skills.

"Curiosity is the number one skill above all else," Echeverria said. "If you find those curious people able to tell stories and explain how they can extract value from data, that's who you want. They understand the data itself, and understand the domain they are working in. The easiest way to convince someone they need data scientists is to talk with them."

Data scientists often have backgrounds in mathematics, advanced computing, visualization, data warehousing, statistics and other fields, and skills in the technologies and tools that tame and process big data, like Hadoop, cloud computing and data visualization tools.

Yet few, if any, individual data scientists are proficient in all of these categories, and such a data superstar would no doubt be in high demand.

The better bet, said Dr. Calvin Andrus, Innovation Officer at the Central Intelligence Agency's Office of the CIO, is to hire a high-quality team of data experts.

Andrus outlined a challenging intelligence effort that his agency recently undertook, in which a whole team of data scientists combined their skillsets to manage and complete a mission that would have been impossible for any individual.

"The best data scientist in the world could not have solved that problem," Andrus said. "We put together a data science team. You want a team without having to find the one data scientist who will save us all."

Andrus said he'd "like to see more science in data science," and it certainly seems a likely scenario. Running a MapReduce job in Hadoop used to be kind of a big deal, but now that's relatively routine -- which again emphasizes why curious data scientists might hold the keys to driving more insights from big data. "At some point, I hope data science is a given," said Michael Parks, Solutions Architect and EMC.

It seems well on the way.