Defense Innovation Board Adopts AI Testing, Digital Workforce Recruitment Resolutions

The Defense Innovation Board convened for its fall public meeting Tuesday and approved resolutions for two key federal technology issues in addition to broadening its work on space. 

The board, which is comprised of national security technology innovators, formed a new space subcommittee to support the Space Force and heard from Michael Kratsios, acting undersecretary of defense for research and engineering, and U.S. chief technology officer. 

But the need for better testing protocols for artificial intelligence systems and strategies the Defense Department could adopt in order to attract digital talent took center stage at the meeting. The board adopted resolutions after robust discussions for both issues.

Challenges in AI Testing 

No proven methods for testing and evaluating nondeterministic AI systems—meaning less predictable, more adaptable AI systems—exist. Daniela Rus, a roboticist with the Massachusetts Institute of Technology, said it is critical to have strong procedures for testing, evaluation, verification, and validation, or TEV/V, of artificial intelligence in order to create enough confidence in the technology to deploy it. 

“The department has been articulating the importance of accelerating the deployment of these systems,” Rus said, citing DOD’s adoption of the board’s AI ethics principles. “We have seen a lot of efforts in developing AI accelerator programs that will take the latest and greatest advancements in AI from research organizations and map them into processes and procedures for the department. We hope to have these in place, but in order to get there we need to have rigorous, robust procedures for testing.”

The main reason testing for these types of autonomous systems is so challenging is uncertainty. Board member Danny Hillis, a pioneer in parallel computing, said uncertainty comes in three directions: from the function, the inputs and the outputs. Hillis suggested the board should use these three areas of uncertainty to guide its thinking when it comes to providing recommendations for TEV/V. 

The resolution adopted by the board argues DOD must develop its own TEV/V solutions as soon as possible, rather than wait for external solutions, in order to be ready to deploy AI systems in the short term. The board’s science and technology subcommittee hopes to have two reports—one for a backgrounder and another for recommendations—on TEV/V for AI by December of this year. 

“Without a strong push for education and training on this topic and a diverse range of testing programs at the developmental and operational levels, DoD will have difficulty assessing its current TEV/V processes and determining next steps to improve its AI TEV/V capability,” the resolution reads. 

Competing for Digital Talent 

Later in the meeting, the DIB turned its attention to workforce issues. Jennifer Pahlka, a founder of the U.S. Digital Service and Code For America, led the group’s discussion on competing for digital talent. Pahlka said the coronavirus pandemic and remote work trends could help the department attract talent if it develops new strategies to help it compete with the private sector. 

“As private sector remote work trends are changing how employers compete for digital talent, DOD has the opportunity to take advantage of these trends and be more competitive for civilian talent in this new environment,” Pahlka said. 

DOD and the federal government in general struggles to fill talent gaps for several reasons, including long hiring timelines. A recent report by the Partnership for Public Service found the average hiring timeline for the federal workforce is 98 days, or more than twice the private sector average. 

The paper DIB released to accompany the discussion detailed five recommendations for what to do to attract digital talent. Overall, DOD should develop strategies to maintain a remote and distributed workforce even beyond the pandemic. Pahlka added that though the recommendations focus on attracting digital talent, she hopes the same principles outlined can be expanded across the workforce. 

In the past, common wisdom said the Pentagon couldn’t do mass telework. Since the onset of the coronavirus pandemic, DOD had to adapt, and fast. Lisa Hershman, the chief management officer for the Defense Department, said in July the pandemic “shattered the myth” DOD couldn’t support remote work. 

According to the DIB’s report, DOD should now focus on expanding its IT infrastructure and make sure it has the tools it needs to maintain remote work as well as expand the agency’s capabilities to do classified work remotely. 

The report also recommends DOD work on improvements to the remote hiring process, prioritize changing the agency’s culture around remote work and “consider dedicated remote work pilot programs to recruit and fill critical civilian technical talent gaps at priority organizations.”

“The subcommittee believes the DOD is really at an inflection point for talent management,” Pahlka said. 

Pahlka and three other members of the DIB including former Google CEO Eric Schmidt ended their terms on the board. Member terms last four years.