Say Freeze? FBI's Facial Recognition Accuracy Unmeasured For Three Years, Warns Watchdog

The FBI still has not assessed whether its facial recognition systems meet privacy and accuracy standards nearly three years after a congressional watchdog — the Government Accountability Office — raised multiple concerns about the bureau’s use of the tech.

Since 2015, the FBI and other law enforcement agencies have used the Next Generation Identification-Interstate Photo System, which uses facial recognition software to link potential suspects to crimes, pulling from a database of more than 30 million mugshots and other photos.  

In May 2016, the Government Accountability Office recommended the FBI establish checks to ensure the software adhered to the Justice Department’s privacy and accuracy standards, but according to a report published Wednesday, the bureau has yet to implement any of the six proposed policy changes.

GAO added every measure to the Justice Department’s list of “priority open recommendations,” though DOJ officials previously disputed whether four of the six policies are necessary.

“By addressing these issues, DOJ would have reasonable assurance that their [facial recognition] technology provides accurate information that helps enhance, rather than hinder, criminal investigations,” Gretta Goodwin, GAO’s director of justice and law enforcement issues, told Nextgov. “Even more, DOJ would help ensure that it is sufficiently protecting the privacy and civil liberties of U.S. citizens.”

Auditors recommended the bureau test the accuracy and effectiveness of the NGI-IPS system at least once a year and make improvements as necessary, but FBI officials said such reviews were unnecessary because “no users have expressed concerns” with the system. However, annual operational assessments are mandatory under FBI, Justice Department and Office of Management and Budget policy, regardless of user feedback, auditors said.

GAO also urged the bureau to test the system’s false positive rate—how often it incorrectly identifies someone as a potential suspect—but officials have yet to do so. The FBI also relies on a handful of facial recognition systems developed by other state and federal law enforcement agencies to conduct investigations, but GAO found the bureau still isn’t measuring whether those tools meet its accuracy standards either.

“Until FBI officials can assure themselves that the data they receive from external partners are reasonably accurate and reliable, it is unclear whether such agreements are beneficial to the FBI, whether the investment of public resources is justified, and whether photos of innocent people are unnecessarily included as investigative leads,” auditors wrote.

The bureau also failed to implement a pair of GAO recommendations for improving the transparency of its facial recognition operations and determining whether its practices respect individuals’ privacy rights.

The report comes as the FBI ramps up its use of facial recognition in criminal investigations. Last year, the bureau started testing a facial recognition tool developed by Amazon as a way to sort through the massive amounts of surveillance footage it collects during investigations. Civil rights groups raised concerns about the accuracy of the Amazon software after it incorrectly matched 28 members of Congress to criminal mugshots.