Social Science

Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 3, Stable Diffusion XL). We identify five distinctive properties of open foundation models (e.g. greater customizability, poor monitoring) that mediate their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work supports a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks.

Response of the AI Policy and Governance Working Group to the Envoy on Technology Call for Papers on Global AI Governance. The AI Policy and Governance Working Group, convened by the Institute for Advanced Study, submits this white paper in response to the United Nations Secretary-General’s Envoy on Technology’s Call for Papers on Global AI Governance.

Comment of the AI Policy and Governance Working Group on the NTIA Request for Comment on Dual Use Foundation Artificial Intelligence Models with Widely Available Model Weights Docket NTIA-240216-0052 The AI Policy and Governance Working Group recommends that the Biden-Harris Administration call for the development of a range of practical approaches to open foundation model release with accompanying case studies and pilot studies developed with relevant stakeholders.