NTIA advocates open AI model weights in report to White House

The National Telecommunications and Information Administration delivered its final report on how the U.S. government handles policy related to dual-use artificial intelligence foundation models — which produce multipurpose tech that have potential for both benefit and harm — recommending that the Biden administration embrace open-source code to continue democratizing and innovating in AI.

Sent to the White House last week, the report is the result of President Joe Biden’s expansive October 2023 executive order on artificial intelligence. After feedback from an initial public comment period, NTIA found that the weights characterizing all-purpose foundation AI models should be publicly available. 

Weights in AI software are mathematical operations that impact the output of a given dual-use AI or machine learning model. Following comment review and further research, NTIA determined that open-source weights will keep advanced AI solutions accessible to a broad demographic.

“The openness of the largest and most powerful AI systems will affect competition, innovation and risks in these revolutionary tools,” said Alan Davidson, assistant secretary of Commerce for communications and information and NTIA administrator, in a press release. “NTIA’s report recognizes the importance of open AI systems and calls for more active monitoring of risks from the wide availability of model weights for the largest AI models. Government has a key role to play in supporting AI development while building capacity to understand and address new risks.”

Within the report, NTIA identifies several actions for the U.S. government to take, namely in regards to monitoring risks. The report calls for a new federal government program focused on collecting and evaluating evidence of open-weight capabilities, risks and benefits.

In terms of collecting evidence, NTIA envisions agencies performing research into powerful AI models and their uses and developing a series of risk-specific indicators. These indicators would essentially act as warning signs for future risks presented by open foundation models. Some examples the report offers are the number of parameters –– or variables that help models make predictions –– training data, model architecture, input data and more.  

Following the collection and evaluation of evidence gathered from open source models, the report recommends acting on the assessments, which could include implementing risk mitigation protocols or outright restricting access to a given model. NTIA also advises generating a series of policy actions for risk scenarios. 

This framework would help set possible policy action. 

“Open foundation model capabilities and limitations are evolving, and it is difficult to extrapolate their capabilities, as well as their impact on society, based on current evidence,” the report reads. “Further, even if we could perfectly extrapolate model performance, quantifying the marginal risks and benefits is extremely difficult. For these reasons, our analysis favors taking steps to develop the evidence base and improve research techniques, as we address in our policy recommendations.”

The report notes that it offers “relatively little insight” into future dual-use foundation models with widely available weights, but still recommends a strong monitoring framework and protocol to help the government determine individual model capabilities. 

It also cited the need for ongoing federal research, such as that conducted by the National Institute of Standards and Technology, the U.S. Copyright Office and the Department of Energy’s Frontiers in AI for Science, Security and Technology as helpful to gauge a holistic downstream impact of open-weight foundation models. 

NTIA also recommended that the U.S. government "maintain a portfolio of risk cases, including unlikely risks and sociotechnical risks, that might arise from future open foundation models."

Some industry players have responded positively to the final report. IBM, who submitted a comment in the draft discussions, advocated for open weight models. 

“At IBM, we believe in an open innovation ecosystem and see the immense value of community-built technology as the best way to grow AI responsibly and mitigate potential misuse,” Darío Gil, IBM senior vice president and director of research, told  Nextgov/FCW. “That’s why we’re pleased to see NTIA’s open weight AI models report and the government’s recognition of the benefits of open-source AI innovation. This report should galvanize government focus on open innovation and accelerate investment in developing and adopting open-source AI resources.”