Measure of Success

Wags may call "business intelligence" an oxymoron, but for a growing number of federal agencies it's no joke. The term represents a class of serious technology tools that can help organizations operate more efficiently and demonstrate their success to watchdogs at the Office of Management and Budget.

OMB, along with the Office of Personnel Management, was charged two years ago with issuing performance score cards to rate the efficiency of federal agencies in five key areas, ranging from financial performance to personnel management. Known as the President's Management Agenda (PMA), this Bush administration initiative seeks to control costs while creating a more responsive and better-managed government (see "How the PMA works," below).

Accountability and management efficiency are honorable goals, but achieving them isn't easy. Baseline rankings of more than 27 federal agencies in the fall of 2001 showed that none achieved across-the-board success. Indeed, most received unsatisfactory ratings for all five evaluation categories.

In today's political climate, getting poor PMA scores is about as much fun as flubbing your college SATs. Although the PMA doesn't specify direct penalties for unsatisfactory score cards, the system is designed to shame managers into compliance, with the long-term implication that promotions and advancements will be harder to obtain.

"There's a lot of pressure to perform," said Mike Sade, the Commerce Department's director of acquisition management. "The PMA is the focus not only at the OMB, but also on [Capitol] Hill."

No wonder senior managers are looking for any help they can get to become PMA successes. Increasingly, they're turning to business intelligence software, which for years has given private industry a way to sort and assemble the arcane details of an operation into a bigger picture that helps them monitor performance and uncover ways to achieve new efficiencies.

"My experience at the [Defense Department] is that [business intelligence] is pervasive if not ubiquitous," said Dan Marion, managing director of systems integrator BearingPoint Inc.

Business intelligence meets the PMA

Business intelligence can address PMA requirements in a number of ways, including:

* Providing managerial accounting information that combines financial data with other measurements, such as quality and customer satisfaction.

* Tracking actual performance against target goals.

* Pulling data from multiple sources into a single repository to meet consolidated e-government requirements.

* Managing human resources.

"The PMA is starting to bring more attention" to business intelligence, said Steve Williams, president of DecisionPath Consulting, a technology consulting firm based in Gaithersburg, Md.

For example, the Interior Department uses technology from Santa Clara, Calif.-based Brio Software Inc. and is testing a workforce planning module from the SAS Institute Inc. to meet the PMA's strategic management of workforce focus areas.

"We have to be able to determine where our organization is now and where we think be it will be in five to 10 years," said Beres Muschett, human resources information systems manager in Interior's Office of Personnel Policy.

The agency is applying the SAS software to predict future staffing needs for wilderness firefighters. "It's helping us do succession planning, since the average age of federal employees means many of these folks are within 10 years or less of retirement age," he said.

For the past four years, the Coast Guard's Aircraft Repair and Supply Center in Elizabeth City, N.C., has been using tools from Cognos Inc. for financial management. Better cost monitoring has helped the department eliminate waste, said Carl Webster, project manager for the decision support system.

"In the past, we never had timely information about where the parts were in the shipping chain," he said. So if a mechanic didn't receive an order, he'd assume it was lost on its way from the manufacturer and reorder the part.

"He'd keep doing that until the part showed up," Webster said. But when original orders were simply delayed in shipping, and not lost, the center would find itself with redundant parts.

Now, thanks to a real-time link between the Cognos software and shipping companies' Web sites, mechanics can create reports that show where each order is in the supply pipeline.

"This tells us exactly where each item is in real time," Webster said. By avoiding the costs of redundant parts, the center saved half a million dollars in one year, he added.

Vendors are also discovering the PMA connection. "Cognos now actively promotes [business intelligence] as a way to achieve [success] ratings on PMA score cards," said Terence Atkinson, the company's director of public-sector solutions. "My impression is that most sales of our tools [in the federal market] have to be in some way tied around PMA. It's part of the criteria on which agencies evaluate products."

Business intelligence parts

Rather than being a single, well-defined class of technology, business intelligence encompasses a wide range of products, including sophisticated data repositories called data warehouses or their smaller, more focused cousins called data marts. Each stores and manages megabytes' or terabytes' worth of information that can be sliced and diced to reveal various measures of an agency's performance.

Business intelligence middleware, another behind-the-scenes component, is software that gathers data from individual applications and databases throughout an organization and feeds it into the centralized marts and warehouses. Through this process, middleware products solve communications and data format incompatibilities that make it otherwise impossible for disparate systems to talk to one another.

Graphics-rich desktop applications, — including online analytical processing tools for technically savvy strategic analysts or decision support systems for less technical business managers — pull information from central warehouses. Managers can then manipulate the information for reports and "what if" analyses to determine how the organization is performing at any given moment and to identify emerging trends or potential operational problems.

But business intelligence projects also have their share of challenges. "The pain point is integrating data, especially when there are legacy mainframes, in order to get one view of the organization," said Sanjay Poonen, senior vice president of Informatica Corp., a Redwood City, Calif., middleware vendor.

Data integration and the construction of data warehouses and data marts can quickly add up to hundreds of thousands or millions of dollars, depending on the size of the initiative. To combat high costs and installation complexity, vendors are now pushing customers toward business intelligence platforms, rather than individual software modules.

By adopting larger, enterprise architectures with a common platform of compatible data formats and communications pipelines, agencies can eliminate stand-alone systems that don't provide a global view of the entire organization.

"The promise of [business intelligence] is not simply to create a score card, but to apply the platform to solve business problems," said Jeff Babcock, vice president of SAS. "Some folks want to do more than just comply with the PMA. They're interested in creating better business processes." Along with SAS and Informatica, Cognos, IBM Corp., Oracle Corp., SAP AG and Microsoft Corp. offer platform components.

First steps

Partly because of their cost and complexity, most federal agencies aren't launching large-scale business intelligence projects just to address the PMA but instead are finding ways to expand or enhance existing systems to address the initiative's requirements.

"Agencies are looking for tools to satisfy specific needs, such as how to better track their various programs to better understand how they're performing against the goals they want to achieve," Cognos' Atkinson said.

Commerce's procurement division has been conducting performance measurements for seven years, since before the PMA existed. Officials recently started using score card technology from SAS to automate the data collection and reporting processes.

The score cards track such areas as employee satisfaction, cost-to-spend ratios, and the educational and training levels of the procurement workforce. In time, this information will have a direct bearing on the department's PMA efforts.

"Part of the objective is not to just collect financial data, but down the road collect more of the performance data related to the PMA," Sade said.

To accomplish this, Commerce officials had to re-engineer the entire financial reporting system to create a single entity that collects and merges data from across the entire organization. They are now in the process of layering score card technology on top of the financial system.

Score cards will arrive first for the procurement department, and later for human resources, administrative support and other areas.

The upshot of all this work is better strategic data. "When you're talking about [business intelligence], you're also talking about how you manage your organization," said Jim Taylor, Commerce's deputy chief financial officer. "Technically, we could meet the requirements of the PMA by just getting the performance data integrated, which we were already doing. But the chief operating officer and the CFO still would not have had the accurate and timely financial information they were looking for.

"So we're bringing everything together by utilizing data warehouses in the field, [Web-based] portal technology and a desktop instrument panel for senior management. They'll have drill-down capability, and combined information about financials, procurement and human resources, all in one desktop screen."

With just a few mouse clicks, senior managers will be able to compare each bureau's spending activities, how much of their budget has been used and staff vacancy levels, including any lag times in hiring replacement workers, Taylor said. "Today, we can get that raw data, but we can't compare it across the board," he added.

Data visualization tools won't only help top executives. Commerce officials plan to offer business intelligence tools to managers in all of its agencies to give them the same "what if" capabilities. "How you present data has a lot to do with how you interpret data," Sade said. "Looking at raw data in a table is a lot different than looking at it in a chart or a graph. We want our bureaus to see exactly what our senior managers see, to share data so they can respond to any questions."

Know your needs

Although business intelligence offers a lot of potential to help agencies fulfill their PMA requirements, experts warn against becoming enamored of the technology's many bells and whistles.

"The selection process has to be very structured," consultant Williams said. "Don't get into a features and functions shootout among vendors. [Business intelligence] vendors have highly experienced salespeople who are experts at trying to convince you their product is the best."

A better approach is to decide what specific tasks you want a business intelligence system to perform for the organization and then challenge vendors to prove their products measure up. A careful evaluation like this is the best way to keep the wags at bay.

Joch is a business and technology writer based in New England. He can be reached at ajoch@monad.net.