A dark grid of evaluation tiles with green and blue highlights, representing item-level life sciences AI benchmark results.

Model Evaluation · · 8 min read

What BioBasicsBench means for life sciences AI teams

Acumen Editorial · Updated November 20, 2026

On November 6, OpenAI released BioBasicsBench, built with the enzyme-design startup Ligo Biosciences. It asks a narrow question that most science benchmarks skip. Can a model read raw experimental data, catch the errors in it and say when the evidence is too thin to support a conclusion?

Published figures are clean. Lab data is not. As OpenAI put it, "everyday wet-lab work involves failed experiments, ambiguous images and conflicting observations." That is the work R&D teams want AI to help with, and it is the work models handle worst.

What is BioBasicsBench?

According to R&D World's report, Ligo scientists turned data from their own experiments into 100 questions. The published examples come from DNA cloning: colony PCR, agarose gel interpretation and negative controls. Each question carries detailed scoring criteria that reward accurate measurement, sound reasoning and stated uncertainty.

Every question went through three reviews: a second Ligo scientist, a PhD-level scientist at an outside wet lab and a final internal check. OpenAI ran seven of its own models across 37 model-and-setting combinations, five attempts per task, for 18,500 graded answers.

At the highest reasoning setting, GPT-6 Astra scored 66.7% and GPT-6.1 Sol 62.0%. Older models landed between 31% and 44%. Astra used about half the output tokens of GPT-5.6 Sol at the same setting.

What do the results actually tell R&D teams?

Three things, and the headline score is the least important.

Correct observation, wrong conclusion. The most telling failure in the report: on one colony PCR task, Astra sorted the gel lanes correctly, then picked colonies that still carried the original plasmid. An exam-style benchmark would have scored that as mostly right. A lab would have lost a week.

Small counting errors compound. Astra overcounted one plate's colonies by about 12%. That error carries straight into the transformation-efficiency estimate. Numbers that look plausible are the hardest to catch.

The ceiling is not the floor. OpenAI itself says higher scores don't yet show reliable lab judgment. The benchmark covers 100 tasks, compares only OpenAI models and has not been independently replicated. Treat it as a signal about where models break, not a ranking to buy against.

How does BioBasicsBench compare with other science benchmarks?

It is the third serious expert-built life sciences benchmark this year. Each tests a different slice of R&D work.

BenchmarkBuilt byWhat it testsSizeReported top result
BioBasicsBench (Nov 2026)OpenAI with Ligo BiosciencesReading raw wet-lab data, spotting errors, stating uncertainty100 tasks66.7% (GPT-6 Astra)
LifeSciBench (Jun 2026)OpenAI with 173 PhD-level scientistsApplied research workflows across seven domains750 tasks, 19,020 rubric criteria36.1% pass (GPT-Rosalind)
DrugDiscoveryBench (Jun 2026)Scale Labs with PhyloAgentic computational drug discovery, target ID to SAR82 tasks51.6% (GPT-5.5)

The pattern holds across all three. LifeSciBench found that tasks with attached files or URLs were much harder: GPT-Rosalind passed 28.1% of them against 45.1% of text-only tasks. Exact numeric answers passed only 14.8% of the time. On DrugDiscoveryBench, agents solved many more tasks once given an expert's step-by-step method, which shows how much of the gap is scientific judgment rather than tool use.

Put simply: the closer a task gets to real data and real decisions, the more models fail. And none of these benchmarks uses your assays, your instruments or your failure modes.

Why public benchmarks can't stand in for your own evaluation

Public benchmarks tell you what a model can do in general. Your team needs to know what it does on your work. Three gaps matter most.

Your data looks different. Gel images from your imagers, plate reads from your readers, chromatograms from your columns. Formats, noise and artifacts vary by lab. A model that reads a clean example may fail on yours.

Your failure modes are specific. Contamination patterns, edge effects, batch drift, off-target signals. Your scientists know which mistakes cost the most. A public benchmark weights them evenly or not at all.

Public sets leak. Once a benchmark is published, its items can end up in training data. A private, held-out set is the only way to keep scores honest over time.

How do you build a lab-data evaluation for your own team?

Follow the method the public benchmarks used, on your own material.

1. Pick the decisions. Start from the calls your scientists make daily: is this clone correct, is this assay valid, is this hit real, does this control pass. Each becomes a task family.

2. Use real, messy data. Pull past experiments, including failed runs and ambiguous results. Strip anything confidential or patient-linked.

3. Write reference answers and rubrics. For each task, a scientist writes the correct read, the reasoning and the caveats. Rubric criteria reward accurate measurement and stated uncertainty, and penalize confident wrong conclusions.

4. Review twice. A second scientist checks every item. A third adjudicates disagreements. That mirrors BioBasicsBench's three-review chain.

5. Run it more than once. Five attempts per task, as OpenAI did, shows how consistent a model really is. Report the all-runs pass rate, not the best run.

6. Grade the reasoning, not just the answer. The Astra gel failure shows why. Score the observation and the conclusion separately.

7. Keep it held out and rerun it. Store the set apart from tuning pipelines. Rerun on every model or agent change.

What should life sciences AI leads do now?

Start small and start with risk. Pick the two or three lab decisions where a confident wrong answer costs the most: a mis-called clone, a false-positive hit, a failed control read as a pass. Build 50 to 100 tasks around them from your own past runs. That is roughly the scale of BioBasicsBench, and it is enough to separate models on the work that matters to you.

Then use it before you buy. Run every candidate model or agent platform against the same frozen set, five times each. Compare item by item, not just on the average. Ask each vendor to show you where their system fails, not only where it passes.

Finally, make it routine. New model versions ship every few months. A held-out set your scientists trust turns each release into a one-day check instead of a debate.

Checklist: what to ask a life sciences AI evaluation vendor

  • Who writes the tasks, and do they run the same assays you do?
  • Are tasks built from raw experimental data, including failed and ambiguous runs?
  • Does every task have a reference answer, reasoning and caveats from a domain scientist?
  • How many reviewers check each item, and how are disagreements resolved?
  • Do rubrics score observation and conclusion separately?
  • Is each task run several times, with consistency reported?
  • How is the set kept out of model training and tuning?
  • Do you get item-level results and full provenance for each task?
  • Do you own the set and can you rerun it yourselves?

How Acumen helps

We build private life sciences evaluations with verified scientists: molecular biologists, assay developers, medicinal chemists, bioinformaticians and translational researchers, matched to your modalities and methods. They turn real experimental data into tasks, write reference answers and rubrics, and adjudicate every disagreement. Every set ships with provenance and stays yours.

See our life sciences model evaluation capability, or read how to build a pharma AI benchmark.

Request a sample dataset Schedule a conversation

FAQ

What is BioBasicsBench?

BioBasicsBench is a benchmark OpenAI released on November 6, 2026, built with Ligo Biosciences. It contains 100 questions drawn from scientists' own wet-lab experiments, such as colony PCR and gel interpretation. Each question has detailed scoring criteria for measurement accuracy, reasoning and stated uncertainty, and went through three expert reviews before inclusion.

How did AI models score on BioBasicsBench?

OpenAI tested seven of its models across 37 settings, five attempts per task. At the highest reasoning setting, GPT-6 Astra scored 66.7% and GPT-6.1 Sol 62.0%, with older models between about 31% and 44%. Only OpenAI models were tested, and OpenAI notes the scores do not yet show reliable lab judgment.

How is BioBasicsBench different from LifeSciBench?

LifeSciBench, released by OpenAI in June 2026, has 750 tasks written by 173 PhD-level scientists across seven research workflows and seven domains, graded by 19,020 rubric criteria. BioBasicsBench is smaller and narrower. It focuses on reading raw wet-lab data, catching errors and recognizing when evidence is too ambiguous to support a conclusion.

Can AI reliably interpret raw experimental data yet?

Not reliably. On BioBasicsBench, the top model scored 66.7%, and its errors included reading a gel correctly and then drawing the wrong conclusion, plus a 12% colony overcount. On LifeSciBench, tasks with attached data files passed far less often than text-only tasks. Expert review of model outputs remains essential for lab decisions.

Should we evaluate models on our own lab data?

Yes. Public benchmarks use other labs' data, formats and failure modes, and their items can leak into training sets. A private, held-out evaluation built from your own experiments, with reference answers and rubrics written by your scientists, shows how a model performs on the decisions your team actually makes, and can be rerun on every model change.

Who should write a life sciences AI evaluation?

Scientists who run the same work every day. Molecular biologists for cloning and sequencing tasks, assay developers for screening data, medicinal chemists for SAR, bioinformaticians for analysis pipelines. Each task needs a reference answer and rubric, a second reviewer and a third to adjudicate disagreements. That is the review structure the leading public benchmarks used.

From the Acumen archive. Explore expert data and Healthcare & Pharma.

← Back to Insights