HealthBench: Evaluating Large Language Models Towards Improved Human Health
A comprehensive evaluation benchmark designed to assess language models' medical capabilities across a wide range of healthcare scenarios.
Editor's note
Rubric-graded healthcare conversations. Raw and length-adjusted scores, benchmark subsets, and publisher grading setups are shown separately. Anthropic uses an Opus 4.8 grader; its scores are not a controlled comparison with OpenAI-reported scores.
Published results
Results are separated by evaluation version and source configuration. Scores in different sets are not a single comparable ranking.
Anthropic · HealthBench · raw
Opus 4.8 grader and safety classifiers enabled. Raw score at max effort. Not a controlled comparison with OpenAI's grading setup.
Five trials; Opus 4.8 grader; safety classifiers and refusal fallback to Opus 5 enabled. No tools or customized system prompt. Not a controlled comparison with OpenAI's grading setup.
| Model | Score | Variant | Source |
|---|---|---|---|
| Claude Sonnet 5.5 | 69.4% rubric score | HealthBench · raw · Opus 4.8 grader | Anthropic · Claude Sonnet 5.5 System Card Sections 8.15–8.15.1, pages 137–138 2026-09-28 |
| Claude Opus 5.5 | 68.1% rubric score | HealthBench · raw · Opus 4.8 grader | Anthropic · Claude Opus 5.5 System Card Section 8.15.1, page 213 2026-09-22 |
Anthropic · HealthBench · length-adjusted
Length-adjusted version of the Anthropic HealthBench score, with Opus 4.8 grading and refusal fallback to Opus 5. Separate from raw scores and OpenAI-reported results.
| Model | Score | Variant | Source |
|---|---|---|---|
| Claude Opus 5.5 | 60.6% length-adjusted rubric score | HealthBench · length-adjusted · Opus 4.8 grader | Anthropic · Claude Opus 5.5 System Card Section 8.15.1, page 213 2026-09-22 |
OpenAI · HealthBench · length-adjusted
| Model | Score | Variant | Source |
|---|---|---|---|
| GPT-6.1 Sol | 58.5% length-adjusted rubric score | HealthBench · length-adjusted · OpenAI report | OpenAI · GPT-6.1 Sol System Card Addendum Section 5.1, Table 7 2026-09-29 |
OpenAI · HealthBench · raw
| Model | Score | Variant | Source |
|---|---|---|---|
| GPT-6.1 Sol | 56.7% rubric score | HealthBench · raw · OpenAI report | OpenAI · GPT-6.1 Sol System Card Addendum Section 5.1, Table 7 2026-09-29 |
OpenAI · HealthBench Professional · length-adjusted
| Model | Score | Variant | Source |
|---|---|---|---|
| GPT-6.1 Sol | 64.2% length-adjusted rubric score | HealthBench Professional · length-adjusted · OpenAI report | OpenAI · GPT-6.1 Sol System Card Addendum Section 5.1, Table 7 2026-09-29 |
OpenAI · HealthBench Professional · raw
| Model | Score | Variant | Source |
|---|---|---|---|
| GPT-6.1 Sol | 67.2% rubric score | HealthBench Professional · raw · OpenAI report | OpenAI · GPT-6.1 Sol System Card Addendum Section 5.1, Table 7 2026-09-29 |
OpenAI · HealthBench Hard · length-adjusted
| Model | Score | Variant | Source |
|---|---|---|---|
| GPT-6.1 Sol | 36.2% length-adjusted rubric score | HealthBench Hard · length-adjusted · OpenAI report | OpenAI · GPT-6.1 Sol System Card Addendum Section 5.1, Table 7 2026-09-29 |
OpenAI · HealthBench Hard · raw
| Model | Score | Variant | Source |
|---|---|---|---|
| GPT-6.1 Sol | 33.4% rubric score | HealthBench Hard · raw · OpenAI report | OpenAI · GPT-6.1 Sol System Card Addendum Section 5.1, Table 7 2026-09-29 |
OpenAI · HealthBench Consensus · length-adjusted
| Model | Score | Variant | Source |
|---|---|---|---|
| GPT-6.1 Sol | 96.0% length-adjusted rubric score | HealthBench Consensus · length-adjusted · OpenAI report | OpenAI · GPT-6.1 Sol System Card Addendum Section 5.1, Table 7 2026-09-29 |
OpenAI · HealthBench Consensus · raw
| Model | Score | Variant | Source |
|---|---|---|---|
| GPT-6.1 Sol | 95.9% rubric score | HealthBench Consensus · raw · OpenAI report | OpenAI · GPT-6.1 Sol System Card Addendum Section 5.1, Table 7 2026-09-29 |
Reproduce: inspect_evals implementation