Can privacy exist in a big data world?

The federal government is the biggest data collector in the world, housing increasingly enormous databases on everything from tax records and census information to signals intelligence and scientific research.

Many federal agencies, especially in the defense and intelligence communities, have begun to make use of big data tools and techniques to sift through those gobs of information looking for key insights they might otherwise miss.

In some cases, especially in light of the National Security Agency’s vast surveillance efforts revealed in leaks by former contractor Edward Snowden, big data have generated big privacy concerns in the public. By simply using a government system – or, in the NSA example, even using the Internet or a telephone – users are offering up information that is logged and can be stored indefinitely.

The Department of Homeland Security, for example, continuously monitors the incoming flow of goods, services and people from outside U.S. borders against no-fly and watch lists to ascertain risk and determine threats. According to Alan Bersin, assistant secretary of international affairs and chief diplomatic officer for DHS, U.S. Customers and Border Patrol “has data from every person that crossed over [the border] in the last 10 years or more.”

Users may not sign up to share that information, but it is shared and stored regardless.

“We have kept a bargain with the American people that the data is used only for the purpose with which it was gathered,” said Bersin, speaking at a recent Bipartisan Policy Center event on big data. “Rather than deny the existence of big data, we can institute it in a regime consistent with American values.”

The Privacy Act of 1974 bars the government to using collected data for any purpose beyond that for which it was intended.  Mary Ellen Callahan, former chief privacy officer for DHS, said concerns grow at the federal level when agencies look to wield the shiniest new technology without knowing exactly what data they want and what they plan to do with it. Now a partner at Jenner & Block, Callahan’s job at DHS was to ensure agency policies protected people’s privacy.

“From the big data perspective, what makes me nervous is using it with uncertainty of how to do anything, not finding what you want to do with the data before you get it,” Callahan said. “Big data needs a lot more definition, more parameters.”

Privacy advocates argue against the government’s collection of data simply because the technology and cheaper cost of data storage allow for it.

Greg Nojeim, senior counsel for the Center for Democracy and Technology, said there is “not a logical end” to collecting increasingly large amounts of data. He said privacy should be addressed on the front-end of policies that allow for large-scale data collection; otherwise they are likely to be overrun in the name of collecting data.

“The world we’re moving toward is one of ‘collect everything, put use restrictions on the back end, and that will fundamentally change the relationship we have with state,’” Nojeim said. “It puts the state in a better control position. We get that big data is here and we adapt to it – to things like use restrictions. But I don’t think we have to have a world where every data set that might contain a piece of data ought to be collected.”

Such restrictions on data collection are simply denying the inevitable, argued Stewart Baker, former assistant secretary for policy at DHS.  “It’s striking to me the extent at which we’re fighting the future,” Baker said.

Baker said “sooner or later, we’ll all get used to the idea” of different notions of privacy, and suggested large-scale data collection could not be stopped even if the government tried. Baker argued against “trying to get people to adopt dumb laws aimed at fighting off the future” as a waste of resources and time, and instead said the government should “try to maintain limitations on use and mission.”

It is technology that allows for the collection, analyzing of and storage of vast amounts of data, but technology may also be what ultimately enables privacy and civil rights to exist in the big data era.

Christina Norwich, an engineer for Silicon Valley software company Palantir, said big data efforts and evolving technologies need not come at the expense of privacy.

“Technology can enable these programs to exist and have proper oversight exist,” Norwich said, noting that some innovative private sector companies have installed internal compliance teams with access to “immutable audit logs.” In such an environment, someone is “watching the watchers,” and every action a person has taken on data is viewable.

“From a technical perspective, I don’t think it has to be either/or,” Norwich said.