Fa et al. (ICML 2026)

BioAgent Bench: AI Agent Evaluation Suite for Bioinformatics

End-to-end bioinformatics pipelines (RNA-seq, variant calling, metagenomics) with a robustness twist: stress-tests agents under corrupted inputs and decoy files, LLM-graded for pipeline progress and outcome validity. Finds frontier agents complete pipelines without heavy scaffolding but break under perturbation. ICML 2026.

bioinformaticsagentic-pipelinescodetabular
Status
cataloged
Saturation
partial
Runnable
no
Cost to run
high

Editor's note

End-to-end bioinformatics pipelines (RNA-seq, variant calling, metagenomics) with a robustness twist: stress-tests agents under corrupted inputs and decoy files, LLM-graded for pipeline progress and outcome validity. Finds frontier agents complete pipelines without heavy scaffolding but break under perturbation. ICML 2026.

Results

Verified scores pending — this eval is cataloged; our own inspect_ai reproduction is the next step.

Sources