Port security technology vetting comes up short in scientists' study

A panel of scientists has faulted the Domestic Nuclear Detection Office's performance testing of electronic devices designed to scan trucks as they drive through the nation's ports.

In a partially redacted report released

on Thursday, a National Academy of Sciences committee said the office, created in 2005 as part of the Homeland Security Department, had used methods with "shortcomings that impair DHS' ability to draw reliable conclusions" about the likely performance of Advanced Spectroscopic Portal machines, which are actively being considered as a major purchase for enhancing port security.

ASP monitors, which are designed as backup technology for detecting nuclear and radiological material smuggled in cargo shipments, have had a troubled journey through a procurement process set in motion when the Bush administration became concerned about threats to U.S. ports from foreign-owned shipping containers.

A 2008 Government Accountability Office report warned that the development processes for ASP and a related technology called the Cargo Automated Advanced Radiography System were exceeding cost estimates by $1 billion. Congress in turn required DHS to obtain the scientific community's verdict on how best to ascertain the technology's effectiveness.

The controversy over ASP machines is part of a larger clash over the government's overall strategy for protecting ports. During a September 2010 hearing, members of the Senate Homeland Security and Governmental Affairs Committee criticized DNDO for being behind schedule in releasing an interagency strategic plan to combat nuclear terrorism, and agency officials promised it by the end of the year. Homeland Security Secretary Janet Napolitano signed off on the Global Nuclear Detection Architecture Strategic Plan, and the document, for official use only, was delivered to Congress on Dec. 20, a DHS official said on Friday.

The booklet released this week by the NAS scientists, which addressed testing, evaluation, cost-benefit analysis and deployment of ASP monitors, said, "The physical tests were not and have not been structured as part of an effort using modeling (computer simulations) and physical tests to build an understanding of the performance of the ASPs against different threats over a wide range of configurations and operating environments." It said the nuclear detection office's methods did not produce evidence that ASP systems were a significant improvement over the portal monitors and handheld radiation detectors already used to scan truck traffic.

The report's conclusions were "deeply troubling" to Sen. Joe Lieberman, I-Conn., chairman of the Senate panel. "This is the third time since 2006 that a major DNDO test has produced unreliable results," he said in a statement. "This is also the third time since 2006 that independent reviewers have documented the inherent dangers of asking those responsible for buying new systems to test and evaluate them and determine if they are worth purchasing."

Chris Ortman, a spokesman for DHS, said the nuclear detection office "appreciates the academy's work evaluating the ASP program," and its recommendations "will be taken into careful consideration. While the ASP program is important, it is just one part of our layered approach to security, which includes technology, well-trained personnel and processes that can adapt to evolving threats."