OpenAI

HealthBench: Evaluating Large Language Models Towards Improved Human Health

A comprehensive evaluation benchmark designed to assess language models' medical capabilities across a wide range of healthcare scenarios.

clinicalhealthcaretext
Status
cataloged
Saturation
partial
Runnable
yes
Cost to run
medium

Editor's note

OpenAI's rubric-graded evaluation of medical/health capabilities across realistic healthcare conversations. Model-graded against physician-written rubrics, so scoring cost is nontrivial. Already in inspect_evals.

Results

Verified scores pending — this eval is cataloged; our own inspect_ai reproduction is the next step.

Reproduce: inspect_evals implementation

Sources