Global experts assess risk of AI catastrophes

Researchers from The University of Queensland and MIT FutureTech surveyed 272 AI experts from across the AI industry, academia, government and the community. 

Associate Professor Michael Noetel from UQ’s School of Psychology said the experts judged there to be at least a 10 per cent probability of catastrophic outcomes from AI over the next 5 years.

“They assessed AI risk across 24 categories by answering 3 questions: which AI risks are most severe, who is most vulnerable to them, and who is most responsible for addressing them,” Dr Noetel said.

“The experts said the top priorities should be AI cyberattacks, weapons development, competitive pressures in AI development and governance failure.

“If no action is taken to reduce risk over the next 5 years, experts believe 18 of the 24 risks are more than 10 per cent likely to cause catastrophic outcomes.

“Unless something changes, this is at least a 10 per cent chance of more than one million deaths, more than USD $100 billion in financial losses, or global collapse of democracy and civil rights.” 

The experts were asked to assess the probabilities of 5 levels of harmful outcomes occurring, from minor to catastrophic harms. 

The harms included physical, financial and societal harms such as authoritarian takeover.

The research found that even if pragmatic mitigations were adopted over the next 5 years, 5 of the 24 risk categories were estimated more than 10 per cent likely to cause catastrophic outcomes. 

Dr Neil Thompson, Director of MIT FutureTech, said the expert estimations were incredibly worrying. 

"If we consider other mature areas of technology such as nuclear power or aviation, risks at that level would be treated as intolerable,” Dr Thompson said.

“These findings are a call to arms for those building and governing AI to make sure the worst outcomes do not occur.”

The findings suggest many sectors are exposed to overlapping risks as AI becomes embedded in decision-critical systems. 

The experts also expressed concern that even without catastrophic outcomes, many AI risks such as false and misleading information, overreliance on AI systems and fraud and scams, could reshape business and society.

Dr Peter Slattery from MIT FutureTech said while the public and users of AI models bear the brunt of AI harm, they have limited power to mitigate them. 

“And because AI development is highly competitive, labs and governments have limited incentives to slow down or invest sufficiently in safety,” Dr Slattery said. 

“This is why laws, treaties and other collective-action mechanisms may become necessary.”

The researchers used the Delphi method to collect 3 rounds of expert contributions anonymously. Answers were de-identified and reported in aggregate, limiting the potential for individual interests to influence results.

Read the study.

Collaboration and acknowledgements