Follow Philly’s Lead and Dive into the Big Data Future

“Big data” is quickly making its way into conversations in government . However, it is difficult for government officials to identify what big data means for their own organizations. What are the challenges? How can they take on something new that does not necessarily connect to their core mission? And not least, how should they tackle the issue to respond to requests from the public?

The big data discussion hits government in two ways: First, big data is created by citizens in their daily online interactions using social media either directly with government or talking amongst themselves about issues related to government. As the recently released first guidance for social media metrics for federal agencies shows, government is just now getting into the groove of measuring, interpreting and acting on insights they can potentially gain from their interactions with citizens. The second trend centers around open government and the launch of the federal data-sharing site data.gov , a public website that hosts hundreds of data sets produced by federal agencies.

Originally, the big data discussion started outside of government, but has direct implications for government as more and more agencies, politicians and citizens are using social media to interact with each other. Social networking platforms, such as Facebook or Twitter, allow citizens to directly connect to government agencies and share their immediate sentiments via comments in their own newsfeed. In doing so, they create hundreds of new data points. Conversations between government and citizens, and also among citizens, might not even directly involve government, but could be related to ongoing hot-button issues, upcoming policy changes or cuts in government programs.

Keeping track of potentially thousands of externally created data points published by citizens on a daily basis has become an unmanageable problem that is slowly being addressed in the public sector. Some agencies have shut down the option to leave comments on their Facebook pages, thus reducing the cost to respond and track that data. Others actively pull in citizen input or moved on to other platforms that focus the conversation on a specific problem and streamline the solicitation of targeted responses and input from the public ( see, for example, Challenge.gov ).

The second trend that government agencies are facing is the mandate of the Open Government Initiative to release government data sets in machine-readable format for public consumption. The flagship initiative data.gov has paved the way for state and local governments to respond in a similar fashion. Most recently, New York State has released its own data portal , a website that hosts about 6,500 datasets from state, city and local government sources in the state.

The challenge for public managers is manifold: they have to identify appropriate data sets, clean them, potentially merge them from different databases, and make sure that they do not contain any information that cannot legally be released to the public. To meet these requirements, agencies need additional resources and staff with appropriate skill sets. Beyond the internal organizational challenges, agencies also need to understand how they can open themselves up for third parties who are reusing the data.

Mark Headd, the newly appointed first chief data officer of Philadelphia, recently spoke to my social media class at the Maxwell School and shared his insights into the world of big data in government.  Mr. Headd was appointed through an executive order of Philadelphia Mayor Michael Nutter . He reports directly to the city’s chief information officer and mayor, who made it a political priority to understand and organically implement elements of the open government movement -- an advantage other cities might not have.

Headd describes himself as a data evangelist and an embedded technologist who has the task to discover government data, think about ways to make it available to the public and find a match between the data and external stakeholders who can potentially use the data to create public value. Internally, he is focused on cultural change more than on data analysis issues or technological problems: He aims to convince managers of the potential value their data can have for the public, and to inform them about citizens’ changing expectations. Mr. Headd then facilitates connections between data sources and potential data users outside of government.

Creating a Data Ecosystem

As one of the first Code for America cities in the United States, Philadelphia’s local tech community of civic hackers is motivated to reuse public information and create valuable applications. As opposed to data.gov, where data sets are mostly available for so-called “elite access” -- a small group of highly trained computer specialists and data analysts -- the approach in Philadelphia focuses on data that is not highly specialized and is already publicly available, such as transit data, day care centers, information about flu shot locations, etc. Most people will consume the existing data through web browsers, either on their desktops or mobile phones. Headd describes Philadelphia’s approach to open data as a focus on the “last mile.” By that he means that the city invites civic hackers who recombine the existing disconnected datasets in a mindful way to go beyond mere display of data sets, as it is done on data.gov. The city wants to collaboratively build new mobile phone applications by recombining data.

Events such as “ Code for Philly ” promote collaboration with the local tech community to use data and build new projects that have the potential to create something of value for the public. Philadelphia, much like Boston, Baltimore, and New York City, has a very active civic technology community with programmers who are passionate about their city. Headd’s goal is to capitalize on that passion.

One example of Headd’s success are applications such as CityGoRound.org , a clearing house for applications built around transit data that help users catch trains. The application and code are made available for reuse in other cities by simply plugging in local transit data. Transit authorities agreed to a standard that makes it easy to share existing applications. As a result, the city and its technology stakeholders are collaboratively building an entire ecosystem around government data from which all cities can benefit.

One of the challenges is convincing citizens to reuse the data and make use of the applications, Headd says. One approach Philadelphia has chosen is to advertise the newly created third-party products on public buses (see for example SEPTAlking ). However, the question of endorsing and publicly sponsoring products that were built outside of government is still an unresolved issue.

Another challenge is changing the bureaucratic culture. For Headd, the solution is to lead a conversation about the effectiveness and efficiency of the current use of government data. He shows public managers how they can reduce inefficiencies in responding to a steady stream of Freedom of Information Act requests to release data to citizens or journalists. Requests can be burdensome and labor-intensive to research. Headd works with public managers to identify the five most common data requests, collaboratively release the data and reduce the administrative burden. Employees can simply point requestors to the publicly available dataset and save time and resources.

For example, the Department of Licenses and Inspections receives multiple requests to release data about the number of locations of vacant houses as well as code violations. By releasing the data on a public website, the city allowed developers to create mobile applications, thereby significantly reducing the number of written requests and phone calls. The research activities for similar types of requests are minimized by simply pointing requestors to the new app. Government staff can then focus their attention on the core mission, instead of being distracted by FOIA requests.

Hackathon events have also enabled efficiencies by allowing civic hackers to build a service on top of government data sets -- they are effectively helping themselves instead of having to reach out to government for help.

Headd shared a few insights on how other chief data officers can tackle issues in their own cities: “Nobody wants to be first, so point people to other success stories in other agencies,” he said. He is continually evangelizing about the value of big data, promoting developments elsewhere, which helps people understand the benefits of releasing data. He suggests showing public managers tangible benefits instead of talking about openness or accountability, which can be very difficult to quantify, especially in budget-driven conversations.

The big data applications Headd sees are limitless: Budgets, spending, crime or transit data enable people to see how well city employees are doing their jobs and help educate the public about improvements in services. Most of the new coverage government receives is unfortunately focused on things that go wrong -- big data can change the focus.

Lastly, social media and government data can come together to create more personalized connections with citizens. Philadelphia has identified about 40 processes, such as voting, in which engagement is low and new experiments to increase feedback are needed. The city recently launched an application to pull citizen opinions into the policy-making processes: Textizen.com allows users to send in their feedback by cellphone -- without needing an expensive smart phone to actively participate in the policy-making process. By institutionalizing easy to use tools to which most have access, tools like Textizen can become part of a government’s future planning process and can automatically generate input without hosting town hall meetings, which attract a limited number of participants.

The example of Philadelphia’s success is certainly an outlier: The city is blessed with a unique combination of advantages that other local governments might not have:

• A mandate to reuse public information,
• A technologist who understands managerial, technological and cultural issues in government, and
• A unique tech community with a passion for the city and an interest in innovation.

However, all cities around the United States are invited to simply reuse existing applications without reinventing the wheel on a daily basis. Get going with resources that are already freely available and dive into the future of big data in government.