Diks et al. (SpatialBench team)
SpatialBench-Long: Verifiable Long-Horizon Spatial Biology
24 evaluations spanning CosMx / Visium / Xenium / MERFISH / Slide-seq / histology / lineage data, graded deterministically over controlled vocabularies with chokepoint rubrics. Extremely hard: three model–harness pairs tie for best at just 11.1% (8/72). Directly dbverse-adjacent. Scores are per model–harness pair (harness in the variant).
spatial-biologyspatial-transcriptomicslong-horizontabularimagecode
Status
curated
Saturation
unsaturated
Runnable
no
Cost to run
high
Editor's note
24 evaluations spanning CosMx / Visium / Xenium / MERFISH / Slide-seq / histology / lineage data, graded deterministically over controlled vocabularies with chokepoint rubrics. Extremely hard: three model–harness pairs tie for best at just 11.1% (8/72). Directly dbverse-adjacent. Scores are per model–harness pair (harness in the variant).
Results
| Model | Score | Variant | Provenance |
|---|---|---|---|
| Gemini 3.5 Flash | 11.1% pass rate | Pi harness | paper |
| GPT-5.5 | 11.1% pass rate | Pi harness | paper |
| GPT-5.5 | 11.1% pass rate | OpenAI Codex harness | paper |
Sources
- paper SpatialBench-Long (arXiv:2605.28065)