—
scBench: A Benchmark for Single-Cell RNA-seq Analysis
Evaluates whether models can solve practical single-cell RNA-seq analysis tasks with deterministic grading. Tasks require empirical interaction with .h5ad data files - agents must load and analyze the data to produce correct answers. Covers 30 canonical tasks across 5 sequencing
single-celltranscriptomicstabularcode
Status
curated
Saturation
unsaturated
Reproduction
not yet
Published results
50 results
Editor's note
Single-cell RNA-seq analysis. Current Latch V1 (formerly V2), legacy V0 (formerly V1), original-paper results and Anthropic's SingleCellBench evaluation are separate because evaluation sets, scoring and harnesses differ.
Published results
Results are separated by evaluation version and source configuration. Scores in different sets are not a single comparable ranking.
V1 · current official leaderboard
Current V1 (formerly labeled V2) uses updated evaluation sets and scoring; separate from legacy V0 and paper results. A macro-mean is not a solved-task count. Starred models use Anthropic's LSVP and remain distinct from standard configurations.
| Model | Score | Variant | Source |
|---|---|---|---|
| GPT-6 Astra | 63.9% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6.1 Sol | 63.6% macro-mean pass rate | V1 · Codex · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Astra | 63.4% macro-mean pass rate | V1 · Codex · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6.1 Sol | 62.9% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5* | 61.9% macro-mean pass rate | V1 · Pi · LSVP · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5* | 61.5% macro-mean pass rate | V1 · Claude Code · LSVP · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Sol | 61.5% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Fable 5.1* | 60.5% macro-mean pass rate | V1 · Claude Code · LSVP · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5.5 | 60.3% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Sol | 59.8% macro-mean pass rate | V1 · Codex · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5.5 | 59.2% macro-mean pass rate | V1 · Claude Code · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Grok 4.6 | 59.2% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Sol | 58.8% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Grok 4.7 | 58.5% macro-mean pass rate | V1 · Grok Build · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5 | 58.3% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5 | 57.6% macro-mean pass rate | V1 · Claude Code · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Grok 4.6 | 57.6% macro-mean pass rate | V1 · Grok Build · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.8 | 57.1% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.5 | 57.1% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5 | 56.4% macro-mean pass rate | V1 · Claude Code · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.8 | 56.2% macro-mean pass rate | V1 · Claude Code · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5 | 55.6% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5 | 55.4% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| DeepSeek V4.1 Flash | 54.7% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5 | 53.8% macro-mean pass rate | V1 · Claude Code · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Luna | 53.8% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.7 | 52.5% macro-mean pass rate | V1 · Claude Code · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.7 | 50.8% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Gemini 3.7 Flash | 50.4% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Luna | 50.3% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Luna | 48.9% macro-mean pass rate | V1 · Codex · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Kimi K3 | 47.5% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 4.6 | 41.7% macro-mean pass rate | V1 · Claude Code · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 4.6 | 39.5% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Nemotron 3 Ultra 550B A55B | 36.6% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Gemini 3.8 Flash | 34.0% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Mistral Large 4.0 Preview | 31.3% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Nemotron 3.5 Lightning | 22.9% macro-mean pass rate | V1 · Pi · 195 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
Anthropic report · SingleCellBench
Anthropic's reported configuration; separate from the Latch V2 Pi and Claude Code leaderboard rows. Do not infer identical harnesses or scoring.
Anthropic's reported configuration; separate from the Latch V2 leaderboard. Do not infer identical harnesses or scoring.
| Model | Score | Variant | Source |
|---|---|---|---|
| Claude Opus 5.5 | 61.2% accuracy | SingleCellBench · Anthropic evaluation | Anthropic · Claude Opus 5.5 System Card Section 8.17.2, page 217 2026-09-22 |
| Claude Sonnet 5.5 | 59.1% accuracy | SingleCellBench · Anthropic evaluation | Anthropic · Claude Sonnet 5.5 System Card Section 8.17.2, page 143 2026-09-28 |
V0 · legacy July 2026 leaderboard
| Model | Score | Variant | Source |
|---|---|---|---|
| GPT-5.6 Sol | 62.1% accuracy | Pi harness | benchmarks.bio — scBench verified leaderboard 2026-07-24 |
| Opus 5 | 60.1% accuracy | Claude Code harness | benchmarks.bio — scBench verified leaderboard 2026-07-24 |
Original paper results
| Model | Score | Variant | Source |
|---|---|---|---|
| Claude Opus 4.6 | 52.8% accuracy | default | scBench (arXiv:2602.09063) |
| Claude Opus 4.5 | 49.9% accuracy | default | scBench (arXiv:2602.09063) |
| GPT-5.2 | 45.2% accuracy | default | scBench (arXiv:2602.09063) |
| Claude Sonnet 4.5 | 44.2% accuracy | default | scBench (arXiv:2602.09063) |
| GPT-5.1 | 37.9% accuracy | default | scBench (arXiv:2602.09063) |
| Grok-4.1 | 35.6% accuracy | default | scBench (arXiv:2602.09063) |
| Grok-4 | 33.9% accuracy | default | scBench (arXiv:2602.09063) |
| Gemini 2.5 Pro | 29.2% accuracy | default | scBench (arXiv:2602.09063) |