Anthropic
What makes a safety score meaningful?
Anthropic has published a methods guide arguing that a model safety claim is only as sound as the evaluation behind it. The article recommends defining the harmful behavior being measured in advance and avoiding test sets made only of easy cases.
Its recommendations include collaboration among domain, methods, and technical experts; checking automated graders against human judgment; and documenting results so they can be reproduced.
Read results with their limitations
Even question format or answer order can change scores. Anthropic therefore calls for uncertainty, sampling choices, and limitations to be visible in reports.
This is not a new model or a certification announcement. It is methodological guidance for teams designing and interpreting safety evaluations.
OpenAI
Google