Diks et al. (SpatialBench team)
SpatialBench-Long: Verifiable Long-Horizon Spatial Biology
Long-horizon spatial biology research workflows. Current Latch V1 (formerly V2) contains 21 evaluation records; legacy V0 and paper scores remain separate, not directly comparable rankings.
spatial-biologyspatial-transcriptomicslong-horizontabularimagecode
Status
curated
Saturation
unsaturated
Reproduction
not yet
Published results
43 results
Editor's note
Long-horizon spatial biology research workflows. Current Latch V1 (formerly V2) contains 21 evaluation records; legacy V0 and paper scores remain separate, not directly comparable rankings.
Published results
Results are separated by evaluation version and source configuration. Scores in different sets are not a single comparable ranking.
V1 · current official leaderboard
Current V1 (formerly labeled V2) uses updated evaluation sets and scoring; separate from legacy V0 and paper results. A macro-mean is not a solved-task count. Starred models use Anthropic's LSVP and remain distinct from standard configurations.
| Model | Score | Variant | Source |
|---|---|---|---|
| Claude Opus 5.5* | 40.5% macro-mean pass rate | V1 · Claude Code · LSVP · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6.1 Sol | 39.9% macro-mean pass rate | V1 · Codex · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5* | 39.3% macro-mean pass rate | V1 · Pi · LSVP · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Astra | 39.3% macro-mean pass rate | V1 · Codex · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6.1 Sol | 38.7% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Astra | 37.5% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Sol | 37.5% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5.5 | 36.5% macro-mean pass rate | V1 · Claude Code · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Sol | 36.3% macro-mean pass rate | V1 · Codex · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5.5 | 34.9% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.8 | 33.3% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.5 | 32.1% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Sol | 31.6% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5 | 30.9% macro-mean pass rate | V1 · Claude Code · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Fable 5.1* | 29.8% macro-mean pass rate | V1 · Claude Code · LSVP · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Gemini 3.7 Flash | 29.8% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.7 | 29.8% macro-mean pass rate | V1 · Claude Code · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Gemini 3.8 Flash | 28.6% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5 | 28.0% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Grok 4.6 | 27.4% macro-mean pass rate | V1 · Grok Build · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.8 | 27.4% macro-mean pass rate | V1 · Claude Code · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Grok 4.6 | 26.2% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| DeepSeek V4.1 Flash | 26.2% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.7 | 26.2% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Luna | 25.0% macro-mean pass rate | V1 · Codex · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Grok 4.7 | 24.4% macro-mean pass rate | V1 · Grok Build · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5 | 24.4% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Luna | 23.2% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Kimi K3 | 22.6% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Luna | 22.6% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 4.6 | 22.6% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5 | 20.2% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5 | 20.2% macro-mean pass rate | V1 · Claude Code · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5 | 19.6% macro-mean pass rate | V1 · Claude Code · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 4.6 | 17.3% macro-mean pass rate | V1 · Claude Code · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Nemotron 3 Ultra 550B A55B | 15.9% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Mistral Large 4.0 Preview | 12.5% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Nemotron 3.5 Lightning | 4.8% macro-mean pass rate | V1 · Pi · 21 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
V0 · legacy July 2026 leaderboard
| Model | Score | Variant | Source |
|---|---|---|---|
| GPT-5.6 Sol | 38.9% pass rate | Codex harness | benchmarks.bio — SpatialBench-Long leaderboard 2026-07-24 |
| Opus 5 | 25.0% pass rate | Pi harness | benchmarks.bio — SpatialBench-Long leaderboard 2026-07-24 |
Original paper results
| Model | Score | Variant | Source |
|---|---|---|---|
| Gemini 3.5 Flash | 11.1% pass rate | Pi harness | SpatialBench-Long (arXiv:2605.28065) |
| GPT-5.5 | 11.1% pass rate | Pi harness | SpatialBench-Long (arXiv:2605.28065) |
| GPT-5.5 | 11.1% pass rate | OpenAI Codex harness | SpatialBench-Long (arXiv:2605.28065) |