Center for AI Safety (CAIS)

WMDP: Measuring and Reducing Malicious Use With Unlearning

A dataset of 3,668 multiple-choice questions developed by a consortium of academics and technical consultants that serve as a proxy measurement of hazardous knowledge in biosecurity, cybersecurity, and chemical security.

biosecuritysafetytext
Status
cataloged
Saturation
partial
Runnable
yes
Cost to run
low

Editor's note

Proxy measurement of hazardous knowledge; the WMDP-bio subset is the biology slice, used for unlearning research. A safety benchmark — higher is not 'better'. Include with that framing.

Results

Verified scores pending — this eval is cataloged; our own inspect_ai reproduction is the next step.

Reproduce: inspect_evals implementation

Sources