AI Companies Urged to Pause Development Amid Rapid Advancements

Anthropic is urging leading artificial intelligence companies to coordinate a slowdown in developing advanced AI systems. The company warns that rapid improvements in AI technology could lead to losing human control over these systems. In a recent blog post, Anthropic highlighted the increasing speed at which AI performs tasks, suggesting that pausing its development could be beneficial.

Anthropic intends to work with others through its internal research institute to explore ways to implement a credible slowdown. They have not yet specified the exact measures. Meanwhile, Anthropic’s competitor, OpenAI, suggested a different strategy. In a report, OpenAI emphasized the necessity for democratic governments to establish rules and safeguards, rather than leaving these decisions to private companies.

AI models are becoming faster at completing software tasks autonomously, according to Anthropic. They explain that with sufficient computing power, AI could engage in ‘recursive self-improvement,’ designing its successors. While this can offer advances in fields like science and healthcare, Anthropic also warns of potential risks where humans might lose control.

The warning comes alongside research from the University of Toronto showing AI tools creating a new AI ‘worm.’ This worm can adapt its hacking strategies across devices, revealing vulnerabilities in vast networks. Lead researcher Nicolas Papernot highlighted that even less powerful AI models pose security threats. Papernot’s research underscores the need for understanding that high-value targets are not alone at risk.

Anthropic’s proposal aims to ensure that societal structures and AI alignment research keep pace with AI advancements. Alignment involves ensuring AI technology aligns with human values and intentions. A coordinated approach would allow AI labs to verify compliance among global rivals, preventing any entity from unfairly advancing during a slowdown.

An international mechanism for AI slowdown is necessary, according to Anthropic, because otherwise, less cautious actors could catch up, raising safety concerns. This post aligns with both Anthropic and OpenAI’s preparation to enter the stock market through IPOs. The potential valuation of Anthropic could reach nearly a trillion dollars.

Papernot shared his findings with Canadian cybersecurity authorities before public release. The study highlights how accessible and modifiable open-source AI tools facilitate cyberattacks. Papernot stressed that devices once considered low-risk, like unused laptops, can now be gateways for attacking high-value targets. The low cost of mounting such cyberattacks increases the risk to any internet-connected device.