Diks et al. (scBench team)

scBench-Long: Verifiable Long-Horizon Single-Cell Biology

21 evaluations across five research areas (melanoma CD8 reactivity, RNA+ATAC regulatory inference, human–monkey chimera development, KRAS lung-tumor aging, COVID-19 lung pathology), deterministically graded with trajectory rubrics. Brutally unsaturated: across 1,068 trajectories the best model–harness pair passes just 25.4% (16/63). Your specialty. Scores are per model–harness pair (harness in the variant).

single-celltranscriptomicslong-horizontabularcode
Status
curated
Saturation
unsaturated
Runnable
no
Cost to run
high

Editor's note

21 evaluations across five research areas (melanoma CD8 reactivity, RNA+ATAC regulatory inference, human–monkey chimera development, KRAS lung-tumor aging, COVID-19 lung pathology), deterministically graded with trajectory rubrics. Brutally unsaturated: across 1,068 trajectories the best model–harness pair passes just 25.4% (16/63). Your specialty. Scores are per model–harness pair (harness in the variant).

Results

ModelScoreVariantProvenance
Claude Opus 4.825.4% pass rateClaude Code harnesspaper
Gemini 3.5 Flash22.2% pass ratePI harnesspaper

Sources

  • paper scBench-Long (arXiv:2606.26563)