Diks et al. (scBench team)
scBench-Long: Verifiable Long-Horizon Single-Cell Biology
Long-horizon single-cell research workflows. Current Latch V1 (formerly V2) contains 22 evaluation records; legacy V0 and paper scores remain separate, not directly comparable rankings.
single-celltranscriptomicslong-horizontabularcode
Status
curated
Saturation
unsaturated
Reproduction
not yet
Published results
42 results
Editor's note
Long-horizon single-cell research workflows. Current Latch V1 (formerly V2) contains 22 evaluation records; legacy V0 and paper scores remain separate, not directly comparable rankings.
Published results
Results are separated by evaluation version and source configuration. Scores in different sets are not a single comparable ranking.
V1 · current official leaderboard
Current V1 (formerly labeled V2) uses updated evaluation sets and scoring; separate from legacy V0 and paper results. A macro-mean is not a solved-task count. Starred models use Anthropic's LSVP and remain distinct from standard configurations.
| Model | Score | Variant | Source |
|---|---|---|---|
| Claude Opus 5.5* | 36.9% macro-mean pass rate | V1 · Pi · LSVP · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5* | 32.4% macro-mean pass rate | V1 · Claude Code · LSVP · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5.5 | 31.8% macro-mean pass rate | V1 · Claude Code · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5.5 | 30.3% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Sol | 29.5% macro-mean pass rate | V1 · Codex · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Sol | 28.4% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Sol | 27.8% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Astra | 27.3% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5 | 27.3% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6.1 Sol | 26.7% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Fable 5.1* | 25.6% macro-mean pass rate | V1 · Claude Code · LSVP · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Astra | 25.0% macro-mean pass rate | V1 · Codex · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6.1 Sol | 25.0% macro-mean pass rate | V1 · Codex · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5 | 24.4% macro-mean pass rate | V1 · Claude Code · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.5 | 21.6% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.8 | 21.0% macro-mean pass rate | V1 · Claude Code · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Gemini 3.8 Flash | 19.9% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Gemini 3.7 Flash | 18.8% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Grok 4.6 | 16.5% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.8 | 16.5% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Grok 4.6 | 15.9% macro-mean pass rate | V1 · Grok Build · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5 | 15.9% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 5 | 15.9% macro-mean pass rate | V1 · Claude Code · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| DeepSeek V4.1 Flash | 14.8% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Kimi K3 | 14.2% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5 | 13.6% macro-mean pass rate | V1 · Claude Code · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Luna | 13.1% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Grok 4.7 | 11.4% macro-mean pass rate | V1 · Grok Build · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.7 | 11.4% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 4.7 | 11.4% macro-mean pass rate | V1 · Claude Code · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-6 Luna | 10.8% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 4.6 | 10.2% macro-mean pass rate | V1 · Claude Code · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| GPT-5.6 Luna | 9.7% macro-mean pass rate | V1 · Codex · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Sonnet 4.6 | 8.5% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Claude Opus 5.5 | 6.8% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Mistral Large 4.0 Preview | 3.4% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Nemotron 3 Ultra 550B A55B | 3.0% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
| Nemotron 3.5 Lightning | 0.0% macro-mean pass rate | V1 · Pi · 22 evaluation records | Latch Bio · V1 leaderboard 2026-10-07 |
V0 · legacy July 2026 leaderboard
| Model | Score | Variant | Source |
|---|---|---|---|
| Opus 5 | 41.3% pass rate | Pi harness | benchmarks.bio — scBench-Long leaderboard 2026-07-24 |
| GPT-5.6 Sol | 38.1% pass rate | Pi harness | benchmarks.bio — scBench-Long leaderboard 2026-07-24 |
Original paper results
| Model | Score | Variant | Source |
|---|---|---|---|
| Claude Opus 4.8 | 25.4% pass rate | Claude Code harness | scBench-Long (arXiv:2606.26563) |
| Gemini 3.5 Flash | 22.2% pass rate | PI harness | scBench-Long (arXiv:2606.26563) |