Edison Scientific
BixBench3: Research-Study-Scale Computational Biology
Twenty research-study-scale computational biology tasks with 138 programmatically graded artifacts.
computational-biologybioinformaticscodetabular
Status
curated
Saturation
unknown
Reproduction
not yet
Published results
19 results
Editor's note
Models implement study-scale analysis plans and produce artifacts graded against published references. The displayed metric is mean artifact score, not fully reproduced studies. The public runner uses Inspect; dataset access requires separate approval.
Published results
20-task cohort · artifact grading
Mean artifact score across 20 research-study tasks (138 graded artifacts), not the percentage of entirely reproduced studies. The source's fallback configuration remains separately labeled. Valid-answer rates and confidence intervals are not reported here.
| Model | Score | Variant | Source |
|---|---|---|---|
| claude-fable-5.1 (with fallback) | 51.5% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall claude-fable-5.1 (with fallback) / — / max 2026-09-21 |
| claude-opus-5.5 | 50.2% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall claude-opus-5.5 / — / max 2026-09-26 |
| gpt-6-astra | 49.2% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall gpt-6-astra / — / max 2026-09-07 |
| grok-4.6 | 48.8% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall grok-4.6 / — / max 2026-09-09 |
| gpt-5.6-sol | 48.0% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall gpt-5.6-sol / — / max 2026-08-26 |
| kimi-k3 | 47.0% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall kimi-k3 / — / max 2026-08-26 |
| gpt-6-sol | 47.0% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall gpt-6-sol / — / max 2026-09-26 |
| glm-5.2 | 46.3% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall glm-5.2 / — / max 2026-08-26 |
| claude-opus-4.8 | 46.0% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall claude-opus-4.8 / — / max 2026-08-26 |
| gpt-5.5 | 43.1% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall gpt-5.5 / — / max 2026-08-26 |
| gemini-3.5-flash | 42.1% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall gemini-3.5-flash / — / max 2026-08-26 |
| claude-opus-5 | 40.6% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall claude-opus-5 / — / max 2026-08-26 |
| gpt-6-luna | 36.9% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall gpt-6-luna / — / max 2026-09-26 |
| gpt-5.4-mini | 36.4% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall gpt-5.4-mini / — / max 2026-08-26 |
| gpt-5.4-nano | 27.5% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall gpt-5.4-nano / — / max 2026-08-26 |
| gemini-3.1-pro-preview | 24.8% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall gemini-3.1-pro-preview / — / max 2026-08-26 |
| claude-sonnet-4.6 | 15.3% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall claude-sonnet-4.6 / — / max 2026-08-26 |
| claude-haiku-4.5 | 1.5% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall claude-haiku-4.5 / — / max 2026-08-26 |
| gemini-3.1-flash-lite | 0.0% mean artifact score | Base · max · 20 studies / 138 artifacts | Edison Scientific · BixBench3 · overall gemini-3.1-flash-lite / — / max 2026-08-26 |