The authors stress that "it is imperative that world leaders take seriously the possibility that highly capable general-purpose AI systems—which could surpass human capabilities in many critical domains—will be developed in the coming decade or two" and that "attempts to introduce guiding principles" are not sufficient.

Lack of safety research in AI systems is a major concern for the experts, who estimate that less than 3% of scientific publications on AI address the topic, combined with the absence of "mechanisms to prevent misuse and imprudent use, especially with respect to autonomous systems capable of independent action," write the authors, a list that includes Nobel laureates, Turing Award winners, and leading researchers.

In the paper, titled "Managing Extreme Risks from Artificial Intelligence in the Near Term," the signatories recommend that governments "create specialized, rapid-response oversight institutions" with robust funding, "require much more rigorous risk assessments with mandatory consequences," and that companies "prioritize safety and demonstrate that their systems cannot cause harm."

For the most powerful AI systems, the authors argue that "governments should be prepared to take the lead in regulating," including licensing, "constraining their autonomy in critical societal functions, and halting their development and deployment in response to worrisome capabilities," among other measures.

For the signatories, the risks of AI are "catastrophic," as the technology "is already rapidly advancing in critical domains such as cyberwarfare, social manipulation, and strategic planning, and could soon pose unprecedented challenges to human control."

According to Stuart Russell of the University of California, Berkeley, this consensus paper "calls for strong regulation by governments, not voluntary codes of conduct written by the industry," because advanced AI systems "are not toys."

"To increase their capabilities before we know how to make them safe is utterly reckless. Companies will complain that it is too hard to meet the regulations—that ‘regulation stifles innovation,’" he said, adding that "there are more regulations on sandwich shops than on AI companies."

For Philip Torr of the University of Oxford, if care is taken, "the benefits of AI will outweigh the downsides," but without such care, there is a "risk of an Orwellian future with a totalitarian state that has total control" over humanity.

Another of the authors, historian Yuval Noah Harari, warns that with this technology, "humankind is creating something more powerful than itself, which might escape human control."

Read Also: Por cá, cerca de 72% das empresas de serviços utilizam ou avaliam usar IA (Portuguese version)