Workman et al. (scBench team)
SpatialBench: Can Agents Analyze Real-World Spatial Biology Data?
Real-world spatial biology analysis. Current Latch V1 (formerly V2), legacy V0 (formerly V1), paper results and Anthropic's SpatialBench Verified evaluation are separate; matching names do not imply identical configurations.
spatial-biologyspatial-transcriptomicstabularimagecode
Status
cataloged
Saturation
unsaturated
Reproduction
not yet
Published results
49 results
Editor's note
Real-world spatial biology analysis. Current Latch V1 (formerly V2), legacy V0 (formerly V1), paper results and Anthropic's SpatialBench Verified evaluation are separate; matching names do not imply identical configurations.
Published results
Results are separated by evaluation version and source configuration. Scores in different sets are not a single comparable ranking.
V1 · current official leaderboard
Current V1 (formerly labeled V2) uses updated evaluation sets and scoring; separate from legacy V0 and paper results. A macro-mean is not a solved-task count. Starred models use Anthropic's LSVP and remain distinct from standard configurations.
| Model | Score | Variant | Source |
|---|---|---|---|
| Claude Sonnet 5.5 | 72.8% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5 | 72.8% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5.5 | 70.1% macro-mean pass rate | V1 · Claude Code · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Grok 4.7 | 69.6% macro-mean pass rate | V1 · Grok Build · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5 | 69.6% macro-mean pass rate | V1 · Claude Code · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5* | 69.0% macro-mean pass rate | V1 · Pi · LSVP · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5* | 68.7% macro-mean pass rate | V1 · Claude Code · LSVP · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5 | 68.4% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5 | 68.4% macro-mean pass rate | V1 · Claude Code · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Sol | 68.1% macro-mean pass rate | V1 · Codex · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Fable 5.1* | 67.5% macro-mean pass rate | V1 · Claude Code · LSVP · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Astra | 67.5% macro-mean pass rate | V1 · Codex · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Sol | 67.5% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Sol | 67.5% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6.1 Sol | 67.3% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Grok 4.6 | 67.3% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6.1 Sol | 67.0% macro-mean pass rate | V1 · Codex · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.8 | 67.0% macro-mean pass rate | V1 · Claude Code · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Astra | 66.7% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5 | 66.7% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Grok 4.6 | 66.1% macro-mean pass rate | V1 · Grok Build · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.8 | 65.8% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5 | 65.8% macro-mean pass rate | V1 · Claude Code · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.5 | 64.1% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Kimi K3 | 63.5% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| DeepSeek V4.1 Flash | 63.2% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Luna | 60.9% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Luna | 59.7% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Gemini 3.8 Flash | 59.4% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Gemini 3.7 Flash | 59.1% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Luna | 58.6% macro-mean pass rate | V1 · Codex · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.7 | 56.5% macro-mean pass rate | V1 · Claude Code · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.7 | 54.8% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 4.6 | 53.9% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 4.6 | 53.3% macro-mean pass rate | V1 · Claude Code · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Mistral Large 4.0 Preview | 51.9% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Nemotron 3 Ultra 550B A55B | 47.0% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Nemotron 3.5 Lightning | 29.0% macro-mean pass rate | V1 · Pi · 115 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
Anthropic report · SpatialBench Verified
Anthropic's reported configuration; separate from the Latch V2 leaderboard. Do not infer identical harnesses or scoring.
Anthropic's reported configuration; separate from the Latch V2 Pi and Claude Code leaderboard rows. Do not infer identical harnesses or scoring.
| Model | Score | Variant | Source |
|---|---|---|---|
| Claude Sonnet 5.5 | 72.5% accuracy | SpatialBench Verified · Anthropic evaluation | Anthropic · Claude Sonnet 5.5 System Card Section 8.17.2, page 143 2026-09-28 |
| Claude Opus 5.5 | 72.0% accuracy | SpatialBench Verified · Anthropic evaluation | Anthropic · Claude Opus 5.5 System Card Section 8.17.2, page 217 2026-09-22 |
V0 · legacy July 2026 leaderboard
| Model | Score | Variant | Source |
|---|---|---|---|
| GPT-5.6 Sol | 74.5% accuracy | Codex harness · verified subset | benchmarks.bio — SpatialBench verified leaderboard 2026-07-24 |
| Opus 5 | 71.0% accuracy | Claude Code harness · verified subset | benchmarks.bio — SpatialBench verified leaderboard 2026-07-24 |
Original paper results
| Model | Score | Variant | Source |
|---|---|---|---|
| Claude Opus 4.5 | 38.4% accuracy | default | SpatialBench (arXiv:2512.21907) |
| GPT-5.2 | 34.0% accuracy | default | SpatialBench (arXiv:2512.21907) |
| Claude Sonnet 4.5 | 28.3% accuracy | default | SpatialBench (arXiv:2512.21907) |
| GPT-5.1 | 27.4% accuracy | default | SpatialBench (arXiv:2512.21907) |
| Grok-4.1 | 24.7% accuracy | default | SpatialBench (arXiv:2512.21907) |
| Grok-4 | 22.8% accuracy | default | SpatialBench (arXiv:2512.21907) |
| Gemini 2.5 Pro | 20.1% accuracy | default | SpatialBench (arXiv:2512.21907) |