Justice's tech research arm needs better metrics

Better Performance Measures Needed to Assess Results of Justice's Office of Science and Technology

The Justice Department's law enforcement technology research arm has not adequately measured the success of its programs despite sharp budget increases, General Accounting Office officials said today.

The Office of Science and Technology in the National Institute of Justice aims to improve law enforcement technology through research and development, information dissemination and application of new technologies. The organization's budget and programs have grown in the past several years, but the programs' success is largely unknown, GAO officials said.

"OST has been unable to fully assess its performance in achieving its goals because it does not measure the extent to which it achieves the intended outcomes of its programs," the GAO report stated.

OST's budget grew from $13.2 million in fiscal 1995 to $204.5 million in fiscal 2003, according to GAO. The range of programs has also changed, from mainly law enforcement to include broader public safety technology research and development, GAO officials said.

The Government Performance and Results Act of 1993 and guidance from the Office of Management and Budget direct agencies to establish measures to assess outcomes of programs. OST has developed some intermediate goals to track progress, but officials haven't done enough, GAO officials said.

Although Justice officials developed outcome measures in August 2002 for the fiscal 2004 budget process, those measures didn't gauge results and were not outcome-oriented but rather output-oriented, GAO said.

"We acknowledge that measuring results using outcome measures is difficult, and may be especially so in relation to some of the types of activities undertaken by OST," the report states. "There are strategies available that have been used by other federal agencies to take steps toward assessing the effectiveness of information dissemination and technical assistance efforts."

GAO officials recommended that the Attorney General instruct the NIJ director to reassess the measures used and better focus on outcome. The report also recommends the development of appropriate intermediate measures to determine effectiveness.

In a response to the report from the assistant attorney general for OJP, Justice officials agreed with the recommendations. The NIJ director plans to reassess the measurements. The assistant attorney general also noted, however, that developing outcome measures for R&D projects is particularly difficult.