Anthropic publishes 10 recommendations for designing, running and reporting safety evaluations

Anthropic published a post on its Engineering Blog with 10 recommendations for how to design, run and report safety evaluations. It says such evaluations are critical for understanding whether a model is ready to deploy, which safeguards to build for it, and where to focus training and research.

The post, drawn from Anthropic's own experience and its system cards, borrows from psychology's reforms after its replication crisis, such as preregistration. Among its recommendations:

• Build the team with both domain and methods experts, and run evals in a sandbox without internet access, with real-time monitoring. Anthropic says that after its own incidents it hardened its evaluation environments and paused many evals that give models internet access.
• Measure what you mean to measure. Refusal rate is often the wrong construct: for self-harm conversations, Anthropic reports an "appropriate response rate" judged across a whole multi-turn conversation against an expert rubric, and pairs harmful prompts with benign ones in the same domain to catch over-refusal.
• Prefer multi-turn simulations built from observed behavior. Anthropic says its single-turn evals are saturated, with harmless response rates of 94% to 98% for every recent model, so they serve mainly as regression checks.
• Calibrate graders, keep criteria atomic and binary, and have humans review cases the grader is unsure of.

Anthropic's announcement had 881 points on X on November 13.