5 steps to truly client-centric metrics

Metrics, analytics and dashboards are all the rage today. CIOs across the public sector are feverishly working to build dashboards that embed predictive analytics and track important organizational metrics, known as key performance indicators. And there is growing evidence that real-time data and predictive information are enabling managers to make evidence-based decisions.

While that is all good, we have uncovered a critical issue: CIOs struggle to build metrics that truly reflect the performance of their own unit — the IT department.

Most CIOs are comfortable capturing metrics on the technical infrastructure they manage, including the amount of uptime (or downtime) of servers and applications, the number of help-desk tickets resolved and the time it takes to resolve tickets. Those metrics are easily captured because of the automated nature by which they are reported and embedded in existing business processes.

Yet those metrics provide little value to stakeholders. Internal and external stakeholders expect IT systems to be reliable, secure and operational. Metrics on operations are only important when an agency has suffered a significant setback (i.e., systems being down or compromised) because they show that the event was an aberration.

In most situations, those metrics do little to change the perception of the IT unit being akin to janitorial services or the department that orders chairs for the office.

In the past few months, we have interviewed more than two-dozen CIOs who work in public agencies at all three levels of government — local, state and federal. Our goal was to understand how those CIOs were using metrics to manage their own departments and communicate the value of investments in IT infrastructure, programs and personnel to stakeholders beyond the IT department.

Our interviews have pointed us to an intriguing finding: Innovative CIOs are moving beyond a reliance on technical metrics and designing client-centric metrics. The results have been significant; stakeholders at the agencies and beyond have a more positive view of IT’s contribution to modernization, service delivery and engagement. Here is how they are doing it.

1. Involve clients from the beginning

In the early days of scoping a major IT project or redesigning an existing system, innovative CIOs ask clients to describe how they would evaluate the success (and failure) of the effort. The conversation about metrics helps the IT department focus on the issues that matter most to the client. The discussions also have the unintended, but welcome, consequence of encouraging clients to prioritize their needs and outcomes.

During those early conversations, a plan is created to collect data on the key metrics everyone will use to benchmark future performance. The CIOs we interviewed were quick to point out that data on key metrics does not exist in many cases, and significant effort might need to be expended to collect the data.

The goal of the conversations is to get clients to clearly specify the overall metrics they care about and articulate the process through which the performance and success of the IT project or system modernization will be linked to agency outcomes. Capturing that process over time allows CIOs to see trends and patterns that enable the construction of maps linking IT performance to agency outcomes across a range of projects by type, system and client group.

2. Keep clients informed about progress

As the IT project gets underway, the client is kept informed about its progress, and data is collected about his or her experience with the project. That data is mostly gathered through regular meetings that address what is working well and what is not, how much disruption the project is causing, and what can be done to reduce that disruption. Those discussions are an important aspect of building advocates and evangelists for the value of IT within the agency.

Such conversations also allow clients to discuss and highlight the value of IT in the context that makes the most sense to them: their own business operations and the realization of their goals and objectives.

The IT team captures and reports metrics on a regular basis so that everyone affected by the project, both directly and indirectly, has situational awareness on its progress and the next key milestones. In addition, some of the unintended consequences of metrics are unearthed during those meetings, and adjustments are made.

For example, one CIO said his team tracked how quickly callers’ issues were resolved by the agency’s help desk. Unfortunately, the unintended consequence was that help-desk personnel were closing calls quickly to meet the target but not effectively resolving the problems, which increased the number of calls.

The IT team adjusted the metrics to measure the number of calls required to resolve an issue and the time spent on each call. A client survey was also initiated to capture information on the callers’ experiences.

3. Report on the things that matter

Innovative CIOs are working with their peers to develop “top 10 lists” when it comes to metrics. Those efforts focus on ensuring that the IT department is reporting on the things that matter to the agency’s key strategic objectives. Although many CIOs are just beginning such efforts, we applaud the approach.

Some of them are building indices that combine multiple indicators into meaningful overall metrics. For example, one CIO has integrated more than 10 indicators to come up with an overall score for IT security, while another is piloting an index to measure the IT department’s innovation capacity and contribution.

Furthermore, CIOs are creating plans to measure their performance on key metrics and share that information with stakeholders on a regular basis.

4. Link IT’s performance to the agency’s mission

Leading CIOs are finding innovative ways to prove their value. Consider Results Minneapolis, a public dashboard that links 34 pages’ worth of IT department performance measures to larger city values. One dashboard heading highlights the goal of having customer-focused and well-managed IT services and operations.

Metrics on the number of IT projects in flight, number of IT projects on budget and expenditure per full-time IT employee compared to other departmental employees are visualized as evidence of how well the IT team is performing. The department’s goals strive to push larger city goals forward and establish its worth in concrete terms.

Other CIOs are looking into how to track the IT department’s contribution to projects that are transforming an agency, or how IT is fundamental to building new programs or implementing new policies. CIOs are comparing the percentages of their budgets and resources that are allocated for those efforts versus standard IT maintenance and provisioning of computing resources.

5. Avoid ‘watermelon metrics’

The best CIOs keep searching for true causes of potential issues and avoid what one leader calls “watermelon metrics.” Performance measures can look green on the outside but be red on the inside when they are split open for close examination. Smart leaders listen for hints from clients, employees and even automated technical metrics that signal a problem. Their trust in green light metrics is earned, not assumed.

For example, one CIO joined a department where server uptime looked strong but was measured in isolation. There were small flags from other infrastructure metrics, so she had her team design an end-to-end server-performance metric; the team members were shocked by the poor results they received. If they had not dug deeper, she and her team would not have known they had a watermelon metric and that their client’s experience was as poor as it was.

Carving out time to collaboratively design customized project metrics is a great business decision. Leading CIOs recognize that it is always worth leaving their comfort zone of technical metrics to design performance measures that illustrate the value of IT through others’ eyes. That is how top CIOs garner trust, funding, and opportunities for growth and innovation.

When CIOs make a habit of customization (even if it comes at the expense of technical excellence), they create satisfied clients who become IT evangelists and sell others on the IT department’s worth. Those evangelists make it easier for CIOs to acquire the resources they need to be even more successful next time and perform even more strongly.