Biology-focused AI evaluations

Official results cover the site's 26 visible models, including fine-tuned Longevity-LLMs and untuned base/frontier models. Per-task metrics retain native units and direction; scores are not averaged into a BioInspect index. Model training exposure and task-specific scoring should be considered when comparing specialized models with general-purpose models.

Edison Scientific's successor to LAB-Bench, covering literature, figures, tables, protocols and biological sequences. Official results are presented per subbenchmark, separating base/tools and input modes. SourceQuality was refreshed in March 2026; these are not the original LAB-Bench ProtocolQA results.

Single-cell RNA-seq analysis. Current Latch V1 (formerly V2), legacy V0 (formerly V1), original-paper results and Anthropic's SingleCellBench evaluation are separate because evaluation sets, scoring and harnesses differ.

Real-world spatial biology analysis. Current Latch V1 (formerly V2), legacy V0 (formerly V1), paper results and Anthropic's SpatialBench Verified evaluation are separate; matching names do not imply identical configurations.

Long-horizon spatial biology research workflows. Current Latch V1 (formerly V2) contains 21 evaluation records; legacy V0 and paper scores remain separate, not directly comparable rankings.

Long-horizon single-cell research workflows. Current Latch V1 (formerly V2) contains 22 evaluation records; legacy V0 and paper scores remain separate, not directly comparable rankings.

Official current V1 results cover 106 evaluation records. Scores are macro-mean pass rates, not whole-study replication rates.

Official current V1 results cover 113 evaluation records. Agents make decisions from experimental data without internet access.

Official current V1 results cover 118 evaluation records. Model-and-harness configurations are retained separately.

Official current V1 results cover 100 evaluation records. Evaluates candidate selection, assay evidence and transfer across biological systems.

Official current V1 results cover 100 evaluation records, including target engagement, safety and disease-model evidence.

Official current V1 results cover 100 evaluation records. Unreported model scores are omitted, not counted as failures.

Agentic bioinformatics investigations using raw biological data. Vals AI reports overall accuracy across 90 tasks, averaged over three full-benchmark runs; these are external evaluations, not BioInspect runs.

Models implement study-scale analysis plans and produce artifacts graded against published references. The displayed metric is mean artifact score, not fully reproduced studies. The public runner uses Inspect; dataset access requires separate approval.

Study findings from July–September 2026 are scored by reference agreement. Each condition has three repeats; uncertainty is standard deviation, not a confidence interval. Retrieval restrictions and published knowledge cutoffs reduce contamination risk but do not guarantee unseen answers.

Rubric-graded healthcare conversations. Raw and length-adjusted scores, benchmark subsets, and publisher grading setups are shown separately. Anthropic uses an Opus 4.8 grader; its scores are not a controlled comparison with OpenAI-reported scores.

Expert-authored life-science tasks with rubric-based scoring, designed to capture ambiguity and scientific judgment. The original 750-task preprint scores and OpenAI's later Gold v1 results are separate; a rubric score is not a whole-task pass rate.

Preliminary 15-task preview of Phylo's trace-based evaluation for long-horizon biological data analysis. It scores analytical process—including data handling, method selection, statistical rigor, source reliability, and reasoning—rather than only final-answer accuracy. Results are preliminary and the benchmark remains a work in progress.

OpenAI's June 2026 benchmark for scientific 'research taste': 129 synthetic genomics / quantitative-biology / translational-medicine problems, each pairing a noisy dataset with a target estimand, graded deterministically from a known causal structure. Extremely hard and unsaturated.

Open-answer agentic bioinformatics: agents must run multi-step analyses over real datasets. Scores are low and the open-answer vs multiple-choice gap is large, so treat MCQ numbers with skepticism. Best public agents still miss roughly half the questions.

Phylo's expert-reviewed 50-task subset of BixBench removes, clarifies, or corrects ambiguous questions and flawed ground truths. Scores are substantially higher than on the full benchmark and must not be treated as directly comparable to full-BixBench results.

Biology-research QA spanning literature, protocols, figures, and sequences. The published GPT-6.1 result below is OpenAI's ProtocolQA Open-Ended adaptation, not the original LAB-Bench multiple-choice task or a BioInspect run.

End-to-end bioinformatics pipelines (RNA-seq, variant calling, metagenomics) with a robustness twist: stress-tests agents under corrupted inputs and decoy files, LLM-graded for pipeline progress and outcome validity. Finds frontier agents complete pipelines without heavy scaffolding but break under perturbation. ICML 2026.

Less a single benchmark than an open-source harness aggregating 166 biomedical benchmarks and 75 tools across 9 functional families, decoupling tool exposure / harness mode / context / scoring. Across 12 backbones, tool-equipped agents gain ~15 pts on average over prior SOTA on 8 representative benchmarks. Useful infrastructure to track.

Broad chemistry knowledge and reasoning over 2,786 QA pairs. Chemistry-adjacent to biology (drug discovery, molecular properties); included as a boundary domain. Already in inspect_evals.

Expert / olympiad-level reasoning across physics, chemistry, and biology (160 problems). Only the biology subset is in-scope for us; unsaturated and hard. Already in inspect_evals.

OpenAI's benchmark for replicating 20 ICML 2024 papers from scratch — not biology per se, but an agentic 'AI for science' boundary benchmark relevant to reproducible research workflows. Long-horizon and expensive to run. Already in inspect_evals.

Clinician-authored visual QA over radiology images — a multimodal medical benchmark. Tests vision-language models, not text-only. Already in inspect_evals.

Proxy measurement of hazardous knowledge; the WMDP-bio subset is the biology slice, used for unlearning research. A safety benchmark — higher is not 'better'. Include with that framing.