-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
340 lines (317 loc) · 23.3 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
<!DOCTYPE html>
<html lang="en" class="h-full">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>A Path for Science- and Evidence-based AI Policy</title>
<meta name="description" content="">
<meta name="keywords" content="">
<meta name="viewport" content="width=1040">
<link rel="preconnect" href="https://fonts.googleapis.com/">
<link rel="preconnect" href="https://fonts.gstatic.com/" crossorigin="">
<link href="https://fonts.googleapis.com/css2?family=Open+Sans:wght@300;400&display=swap" rel="stylesheet">
<link href="https://fonts.googleapis.com/css2?family=IBM+Plex+Serif:wght@400;700&display=swap"
rel="stylesheet">
<link href="./assets/css/css2" rel="stylesheet">
<link href="https://fonts.googleapis.com/css2?family=Lato:wght@300;400;700;900&display=swap" rel="stylesheet">
<!-- Twitter Meta Tags -->
<meta name="twitter:card" content="summary_large_image">
<meta property="twitter:domain" content="understanding-ai-safety.org">
<meta property="twitter:url" content="https://understanding-ai-safety.org/">
<meta name="twitter:title" content="A Path for Science- and Evidence-based AI Policy">
<meta name="twitter:description" content="">
<meta name="twitter:image" content="https://understanding-ai-safety.org/assets/image.png">
<!--
<script src="./assets/js/splide.min.js"></script>
<link rel="stylesheet" href="/assets/css/splide.min.css">
-->
<script src="./assets/js/61c2f15e92f56eaa354c18452db280ac.js"></script>
<link rel="stylesheet" href="./assets/css/style.css">
<script>
document.addEventListener('DOMContentLoaded', function () {
new Splide('.splide').mount();
});
</script>
</head>
<body>
<!-- <div style="background-color: #003262">
<div class="constraint">
<div id="header">
<div id="logo-container" style="padding-top: 1rem;">
<a href="./" aria-label="Berkeley Logo">
<img src="./assets/logo-berkeley-gold.svg" width="200" aria-label="Berkeley Logo">
</a>
</div>
</div>
</div>
</div> -->
<div style="border-bottom:3px solid black;">
<div class="constraint">
<div style="text-align: center;">
<h1>
A Path for Science‑ and Evidence‑based AI Policy
</h1>
<p>
Rishi Bommasani<sup>†1</sup>, Sanjeev Arora<sup>3</sup>,
Yejin Choi<sup>4</sup>,
Li Fei‑Fei<sup>1</sup>,
Daniel E. Ho<sup>1</sup>, Dan Jurafsky<sup>1</sup>, Sanmi Koyejo<sup>1</sup>,
Hima Lakkaraju<sup>5</sup>,
Arvind Narayanan<sup>3</sup>,
Alondra Nelson<sup>6</sup>,
Emma Pierson<sup>7</sup>, Joelle Pineau<sup>8</sup>, Gaël Varoquaux<sup>9</sup>,
Suresh Venkatasubramanian<sup>10</sup>,
Ion Stoica<sup>2</sup>, Percy Liang<sup>1</sup>, Dawn Song<sup>†2</sup>
</p>
<p>
<sup>1</sup>Stanford University
<sup>2</sup>UC Berkeley
<sup>3</sup>Princeton University
<sup>4</sup>University of Washington
<sup>5</sup>Harvard University
<sup>6</sup>Institute for Advanced Study
<sup>7</sup>Cornell University
<sup>8</sup>McGill University
<sup>9</sup>INRIA
<sup>10</sup>Brown University
<!-- <br> -->
<!-- <sup>*</sup>Corresponding author -->
</p>
<p style="font-size: 0.85em;">
<sup>*</sup>Several authors have unlisted affiliations in addition to their listed university
affiliation. This piece solely reflects the authors' personal views and not those of any affiliated
organizations, including the listed universities.
</p>
</div>
</div>
</div>
<style>
.main-text div {
margin-left: 3vw;
margin-right: 3vw;
margin-top: 45px;
font-size: 16px;
line-height: 24px;
}
@media (min-width: 1500px) {
.main-text div {
margin-left: 7vw;
margin-right: 7vw;
}
}
</style>
<div class="main-text">
<div class="features-wrapper constraint">
<div class="row">
<div class="col-1"></div>
<div class="col-12 mt-5 paragraph-spacing">
<p><strong>Overview:</strong> AI is a powerful technology that carries both benefits and risks. We
wish to promote innovation to ensure its potential benefits are responsibly realized and widely
shared, while simultaneously ensuring that current and potential societal risks are mitigated.
To address the growing societal impact of AI, many jurisdictions are pursuing policymaking. The
AI research and policy community lacks consensus on the evidence base relevant for effective
policymaking, as has been seen with the debates over California’s Safe and Secure
Innovation for Frontier Artificial Intelligence Models Act (California’s SB-1047). Points
of contention include disagreement about what risks should be prioritized, if or when they will
materialize, and who should be responsible for addressing these risks.</p>
<p> In light of
this, we firmly believe AI policy should be informed by scientific understanding of AI risks and
how to successfully mitigate them. Therefore, if policymakers pursue highly committal policy,
the evidence of the associated AI risks <a
href="https://www.ntia.gov/sites/default/files/publications/ntia-ai-open-model-report.pdf">should
meet a high evidentiary standard</a>. Advancing significant legislation without clear
understanding of risks it intends to address may lead to more negative consequences than
positive outcomes.</p>
<p>We support evidence-based policy and recognize current scientific
understanding is quite limited. Therefore, we recommend the following priorities to advance
scientific understanding and science- and evidence-based AI policy:</p>
<ul>
<li>We need to better understand AI risks.</li>
<li>We need to increase transparency on AI design and development.</li>
<li>We need to develop techniques and tools to actively monitor post-deployment AI harms and
risks.</li>
<li>We need to develop mitigation and defense mechanisms for identified AI risks.</li>
<li>We need to build trust and reduce fragmentation in the AI community.</li>
</ul>
<p>We describe each of these priorities in more detail below. We believe by following these
steps we can pave a more productive path toward robust and responsible AI, anchored in the best
scientific practice and AI policymaking that is evidence based.</p>
<p><strong>First, we need to better understand AI risks.</strong> As AI models rapidly advance in
capability, our level of investment dedicated to research for understanding risks should also
increase. However, our understanding of how these models function and their possible negative
impacts on society <a
href="https://www.gov.uk/government/publications/international-scientific-report-on-the-safety-of-advanced-ai/international-scientific-report-on-the-safety-of-advanced-ai-interim-report">remains
very limited</a>. Because this technology has the potential to be far more powerful than
existing technologies and carries wide-reaching implications, it is crucial to have a
comprehensive understanding of AI risks. Such a comprehensive understanding of AI risks is the
necessary foundation for effective policy. In particular, we recommend extensive and
comprehensive study of a broad spectrum of different risks, including discrimination, scams,
misinformation, non-consensual intimate imagery, child sexual abuse material, cybersecurity
risks, environmental risks, biosecurity risks, and extreme risks. To build understanding, we
recommend such studies apply a <a href="https://arxiv.org/pdf/2403.07918">marginal risk
framework</a>, assessing the additional societal risks posed by AI models compared to
existing technologies like internet search engines. This approach will identify AI's unique
risks and negative impacts on society. Given our current limited understanding of risks, we
recommend policymakers invest resources across academia, civil society, government and industry
to foster research on AI risk analysis. Ultimately, these risk analyses will help inform
decisions on important policy questions such as whether to release a model, how to release it,
and what uses of AI should be permitted.</p>
<p><strong>Second, we need to increase transparency.</strong> Analyzing risks and developing policy
is challenging <a href="https://arxiv.org/abs/2310.12941">without adequate information</a>.
Transparency requirements for advanced models would facilitate risk analysis of capable models
with potentially significant impacts. A number of important questions related to transparency
require further study. One key question is the criteria used in policymaking to determine which
entities and models are in scope. Current policies such as the US Executive Order on Safe,
Secure, and Trustworthy Development and Use of Artificial Intelligence and the European
Union’s AI Act set thresholds based on <a
href="https://arxiv.org/abs/2405.10799">compute</a> used to train an AI model, but there is
currently no <a href="https://arxiv.org/abs/2407.05694v1">solid evidence</a> behind these
numbers. More rigorous studies should be required to establish criteria that strike the right
balance between innovation and risk mitigation. Furthermore, we need to <a
href="https://arxiv.org/abs/2402.16268">study</a> the level of information disclosure that
best balances usefulness and potential overhead for model developers. Key questions for
increasing transparency include: what information should be shared with <a
href="https://arxiv.org/pdf/2404.02675">different parties</a> (e.g. the public, trusted
third parties, the government); what information developers should report about models including
size, a summary of training data and methods, capabilities (such as test results on certain
benchmarks and red teaming practices); and incidents such as model theft, unauthorized access,
and the inadvertent release of model weights. In line with this approach, the establishment of a
<a
href="https://carnegieendowment.org/posts/2023/07/its-time-to-create-a-national-registry-for-large-ai-models?lang=en">registry</a>
for capable models might improve transparency.
</p>
<p><strong>Third, we need to develop an early warning detection mechanism. </strong>While current
model capabilities may not pose devastating consequences, given the rapid pace of technological
development, significantly more powerful and risky AI models are likely to emerge in the near
future. It's crucial to draw on research to establish early warning detection mechanisms as soon
as possible, providing society with more time to implement stronger mitigation and response
measures, and hopefully preventing devastating consequences and the crossing of predefined <a
href="https://idais.ai/#">red lines</a>. The mechanisms should encompass both in-lab testing
and real-world monitoring. In a lab setting, we need to detect and evaluate the dangerous
capabilities of models through rigorous evaluation and red teaming. This involves testing AI
models with adversarial scenarios to uncover potential vulnerabilities or unintended behaviors,
and assessing how the models could lead to significant marginal risks in areas such as
cybersecurity and biochemical weapons. Such evaluation and red teaming analyses can help
identify risks before models are deployed in the real world. After the models are launched, the
mechanism should enable <a
href="https://ai.gov/wp-content/uploads/2023/12/Recommendation_Improve-Monitoring-of-Emerging-Risks-from-AI-through-Adverse-Event-Reporting.pdf">adverse
event reporting</a>, documenting how AI has been used for misuse and what consequences it
has caused in the real world. To this end, continuous real-world monitoring of how misuse of AI
may have caused harm in each application domain such as biotechnology and cybersecurity is <a
href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4634443">crucial</a>. Moreover, we
need to determine to whom these early warnings should be reported and design a <a
href="https://arxiv.org/pdf/2404.02675">responsible report protocol</a>. This will help
ensure that identified risks are communicated effectively among the relevant authorities or
stakeholders.</p>
<p><strong>Fourth, we need to develop technical mitigation and defense mechanisms.
</strong>Investing in research to create these solutions is crucial for effectively addressing a
wide range of AI risks. First, instead of just relying on today’s alignment approaches, it
is important to explore and develop new approaches for building safe AI with the potential for
greater safety assurance. This is a complex challenge, which requires a <a
href="https://www.gov.uk/government/publications/international-scientific-report-on-the-safety-of-advanced-ai/international-scientific-report-on-the-safety-of-advanced-ai-interim-report">multifaceted
approach</a>. Second, in addition to the long-term research that might be required to
develop safer AI models, it is also important to develop <a
href="https://arxiv.org/pdf/2405.10295">defensive approaches or immune systems</a> in
society to reduce the potential negative impacts from misuse of AI technology. For example,
additional defensive systems can involve improving the security posture and defenses of computer
systems against security risks caused by AI misuse.</p>
<p><strong>Fifth, we need to build trust and unite the community by reducing fragmentation.</strong>
Currently, the AI community is heavily <a
href="https://yoshuabengio.org/2024/07/09/reasoning-through-arguments-against-taking-ai-safety-seriously/">fragmented</a>
with a variety of views on approaches to risk and policy: the fragmentation poses a challenge to
scientific consensus that would support evidence-based AI policymaking. One extreme position
calls for strong regulation to mitigate extreme risks, whereas the other extreme calls for no
regulation to avoid inhibiting innovation. We support a third approach, an evidence-based
approach to AI policy, which reduces fragmentation towards finding the best solutions for
fostering innovation while mitigating risks. To achieve this, we recommend the continued
development of <a
href="https://www.gov.uk/government/publications/international-scientific-report-on-the-safety-of-advanced-ai/international-scientific-report-on-the-safety-of-advanced-ai-interim-report">collaborative
research initiatives</a> that <a href="https://www.ias.edu/stsv-lab/aipolicy">bring together
diverse perspectives</a>. Creating platforms for respectful scientific debate and shared
problem-solving in a trusted setting can help bridge the gap between different viewpoints.
Additionally, establishing interdisciplinary research groups that include AI researchers, social
scientists, legal scholars, domain experts, policymakers, and industry representatives can
promote a more inclusive and comprehensive approach to AI development, AI safety, and AI policy.
We hope this approach can also lead to greater <a
href="https://www.un.org/sites/un2.un.org/files/governing_ai_for_humanity_final_report_en.pdf">international
cooperation</a>.</p>
<br />
<h2><strong>Call to action.</strong></h2>
<p>We call upon the AI research and policy communities to proactively work together to advance
science and develop evidence-based policy. </p>
<p>We envision that these communities could
produce a forward-looking design, or <em>blueprint</em>, for AI policy that maps different
conditions that may arise in society (e.g. specific model capabilities, specific demonstrated
harms) to candidate policy responses. By working together on this practical, actionable
blueprint for the future, we can work towards scientific consensus, even when different
stakeholders significantly disagree on how capabilities will evolve or how risks should be
addressed. Such a blueprint would complement current guidance on how AI can be developed and
deployed responsibly such as the National Institute of Standards and Technology Risk Management
Framework <a
href="https://airc.nist.gov/AI_RMF_Knowledge_Base/AI_RMF/Core_And_Profiles/6-sec-profile">use-case
profiles</a>. </p>
<p>A blueprint for the appropriate conditional response under
different societal conditions would organize the discourse on AI policymaking around a concrete
artifact. In turn, even if consensus cannot be reached on the likelihood of particular outcomes
(e.g. how much can AI increase the likelihood of large-scale cyber attacks or disinformation
campaigns), progress could be made towards determining the appropriate course of action.
</p>
<p>We should organize a series of convenings as a forum for sustained multi-stakeholder dialogue
that reflects many different positions, disciplines, and institutions. Through these convenings,
we will develop milestones toward progress to science-based AI policy recommendations:</p>
<ul>
<li>Milestone 1: A taxonomy of risk vectors to ensure important risks are well represented</li>
<li>Milestone 2: Research on the marginal risk of AI for each risk vector</li>
<li>Milestone 3: A taxonomy of policy interventions to ensure attractive solutions are not
missed</li>
<li>Milestone 4: A blueprint that recommends candidate policy responses to different societal
conditions</li>
</ul>
<p>By taking these steps, we will build a strong evidentiary foundation, along with broad
engagement and thoughtful deliberation, for producing better policy.</p>
</div>
</div>
</div>
<br>
<div class="features-wrapper constraint" style="text-align: center;">
<a href="https://docs.google.com/forms/d/e/1FAIpQLSfHUKLiLvPm81UE8aL92-on1Q-LusYLNiRcPM4LscjYdDe3ZA/viewform?embedded=true"
style="font-size: 22px; display: inline-block; padding: 15px 20px; background-color: #007BFF; color: white; text-decoration: none; border-radius: 5px;">
Join here to stay informed and contribute</a>
</div>
<br><br><br>
<style>
body {
font-family: 'IBM Plex Serif', serif;
}
.name {
font-weight: bold;
font-size: medium;
}
.details {
font-weight: normal;
font-size: small;
}
.signer ul {
list-style-type: none;
padding-left: 0;
margin: 0;
}
.signer li {
margin-bottom: 12px;
}
</style>
</div>
<!-- <div id="bottom">
<div id="footer">
<div id="footer-text-container">
<p class="footer-text">
For general inquiries, <a class="footer-text" href="mailto:[email protected]">reach us by
email.</a>
</p>
<p class="footer-text">
© UC Regents 2024.
</p>
</div>
</div>
</div> -->
</body>
</html>