AI giants pledge to act over fears tech is developing too fast | Science, Climate & Tech News

Tech bosses Elon Musk and Sam Altman have backed a call from the head of Anthropic to have permanent third-party reviewers inside “frontier” AI companies.

Chief executive Dario Amodei has written an essay called We Must Pace the Frontier, in which he urges such companies to slow the rate at which they improve the capabilities of AI models.

It comes amid increased fears about the possible misuse of AI, with Anthropic recently revealing that several actors had used its Claude AI models for activities including weapons development, cyber operations, surveillance ​and fraud.

👉Listen to This Is Why on your podcast app👈

Mr Amodei has come up with a three-part plan, the first of which he said his AI company is unilaterally committed to.

In a post on X in which he commits to the “first of these steps”, he writes: “We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.”

Musk, the boss of X which has developed it’s own ‘Grok’ AI, replied to the post, writing: “Dario is right.”

And OpenAI boss Altman wrote on X: “I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks.

“Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”

Mr Amodei said he was concerned that since this summer, AI has been “advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI”.

And he warned that within six to 12 months AI could be capable of leading a “swarm” that could take over the entire internet, among other risks.

Anthropic researcher Jacob Coxon ⁠recently resigned, claiming the “people building AI earnestly believe that it could kill us all by the end of ‌the decade”.

Mr Amodei said he was not calling for stopping model training or technical progress, but making sure companies take enough time to align and ‌safeguard their models, and for third-party evaluators to confirm these steps.

What are his proposed three steps?

• Embedded evaluators. Each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR), whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.

This is the key step for verifiability of any pacing commitments, and has precedent in the banking industry, which sometimes involves regulatory “supervisors” embedded along with employees. Anthropic is unilaterally committing to this step now. We intend this to be part of a broader push to redouble efforts on our safety and alignment work.

• Democratic coordination. Frontier AI companies within democratic countries coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress. Some forms of coordination that would be impactful for pacing are legally challenging, and will require government support.

• Global coordination. The US and other democratic governments attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance.