OpenMOSS
SWE-bench Science — Biology & BME
Coding agents repair biology and biomedical engineering software: a 25-task subset of SWE-bench Science.
biologybiomedical-engineeringcodetext
Status
curated
Saturation
unknown
Reproduction
not yet
Published results
10 results
Published results
Subset: 25 of 119 tasks — 13 biology and 12 biomedical engineering. BioInspect calculates these scores from published task outcomes, not new runs. Only configurations with complete subset evidence are included. Source updated 2026-09-10. Download results JSON · Source snapshot
| Model | Score | Variant | Source |
|---|---|---|---|
| Claude-Opus-5 (max) · Claude Code | 68.0% Pass@1 17/25 tasks | Biology + BME · 25 tasks | Official task outcomes · BioInspect subset calculation 2026-09-10 |
| GPT-6 Astra (max) · Codex | 64.0% Pass@1 16/25 tasks | Biology + BME · 25 tasks | Official task outcomes · BioInspect subset calculation 2026-09-10 |
| DeepSeek-V4-Pro (max) · Claude Code | 60.0% Pass@1 15/25 tasks | Biology + BME · 25 tasks | Official task outcomes · BioInspect subset calculation 2026-09-10 |
| Kimi-K3 (max) · Kimi Code | 52.0% Pass@1 13/25 tasks | Biology + BME · 25 tasks | Official task outcomes · BioInspect subset calculation 2026-09-10 |
| Qwen3.8-27B (xhigh) · Claude Code | 48.0% Pass@1 12/25 tasks | Biology + BME · 25 tasks | Official task outcomes · BioInspect subset calculation 2026-09-10 |
| GLM-5.2 (max) · Codex | 44.0% Pass@1 11/25 tasks | Biology + BME · 25 tasks | Official task outcomes · BioInspect subset calculation 2026-09-10 |
| GPT-5.6-sol (max) · Codex | 44.0% Pass@1 11/25 tasks | Biology + BME · 25 tasks | Official task outcomes · BioInspect subset calculation 2026-09-10 |
| Nex N2 (default) · Codex | 36.0% Pass@1 9/25 tasks | Biology + BME · 25 tasks | Official task outcomes · BioInspect subset calculation 2026-09-10 |
| DeepSeek-V4-flash (max) · Claude Code | 36.0% Pass@1 9/25 tasks | Biology + BME · 25 tasks | Official task outcomes · BioInspect subset calculation 2026-09-10 |
| Intern-S2-Preview-397B (max) · Claude Code | 28.0% Pass@1 7/25 tasks | Biology + BME · 25 tasks | Official task outcomes · BioInspect subset calculation 2026-09-10 |
Sources
- external Official task outcomes · BioInspect subset calculation
- official repository Benchmark implementation and reproduction instructions using Pier
- official task outcomes Published binary task results and complete pass lists
- task domain definitions Appendix B, Table 5: 13 biology and 12 biomedical engineering tasks
- paper benchmark paper