Life Sciences · · 8 min read
How to buy RL environments for life sciences AI
Acumen Editorial · Updated November 22, 2026
The money in AI training data has moved from labels to environments. On November 10, 2026, Mercor announced a $750 million Series D at a $20 billion valuation. It said the funds will go to evals, post-training and more environments where agents learn complex workflows, and it acquired Deeptune, a team that rebuilt hundreds of enterprise applications for agents to practice in.
Nine days later, Isomorphic Labs and MSD announced a research collaboration pairing Isomorphic's AI drug design engine with MSD's proprietary data. Isomorphic's president called those data sets "rich." The message for life sciences teams is plain. Models are getting trained on scientific work, and the quality of the training signal now matters as much as the model.
If you lead post-training, evaluation or R&D AI, you will be pitched environments for biology, chemistry and bioinformatics. Here is how we would buy one.
What is an RL environment?
An RL environment is a sandbox where a model attempts a task, uses tools, and gets a reward based on how well it did. It has three parts: the task, the tools, and the grader. In science, the tools might be a Jupyter notebook, a sequence database or a protocol simulator. The grader is the part that decides whether the model learns real science or learns a shortcut.
NVIDIA's guide to training scientific agents describes reinforcement learning with verifiable rewards, which uses computational checks such as code execution to produce repeatable reward signals. It names Aviary, an open framework of scientific environments covering literature research, bioinformatic data analysis and lab tasks such as molecular cloning, and BixBench, a benchmark of verifiable bioinformatics questions.
Why are life sciences environments harder to build?
Because science does not grade itself the way code does. A coding task can be run 64 times and checked against unit tests for almost nothing. SemiAnalysis notes that a single biological experiment can cost hundreds to thousands of dollars and take hours or days. That makes rewards sparse.
Four problems follow.
| Problem | What it looks like in life sciences | What a good environment does |
|---|---|---|
| Sparse rewards | The true answer is a wet-lab result days away | Scores intermediate steps with expert rubrics and splits tasks into graded pieces |
| Contested ground truth | Published literature disagrees with itself | Uses tasks with answers scientists have verified, and records disagreement |
| Closed data | Pharma holds the most useful data privately | Builds tasks from licensed or client-owned data under clear terms |
| Reward hacking | The model finds a shortcut that scores well but is wrong | Audits high-reward rollouts by hand and patches the grader |
The rubric workaround comes from SemiAnalysis, which suggests rewarding intermediate steps and splitting long tasks into graded parts. Rubrics only help if the people who wrote them know the science.
What makes an RL environment worth buying?
Epoch AI interviewed environment builders and lab researchers in January 2026. Their answers are a useful buying standard.
- Soundness first. Interviewees ranked robustness to reward hacking as the top quality criterion. One researcher put it simply: "high reward must mean the task was actually solved, not hacked."
- Calibrated difficulty. Tasks should not pass at 0% or 100%. Interviewees suggested a floor of about 2–3%, or one success in 64–128 rollouts, and retiring tasks near a 70% pass rate.
- A spread of difficulty. A mix of hard, medium and easy tasks gives the model a gradient to climb.
- Shared skills. Tasks should build on common skills so learning carries over.
- Realism. Tools and data must behave like the real thing. Epoch warns that many quickly built clones are buggy and useless.
- Domain experts over ML generalists. One founder told Epoch that domain knowledge matters more than ML skills for writing tasks.
How should you test an environment before you buy?
Run a pilot on a slice of the environment before you sign. We would check the following.
- Ask for 50 tasks with their graders, reference solutions and author credentials.
- Run your current model and log pass rates per task. Flag anything at 0% or 100%.
- Have two independent PhD scientists in the domain grade 100 rollouts blind. Compare their scores with the environment's reward.
- Pull the 20 highest-reward rollouts and read them. Look for answers that score well but are scientifically wrong.
- Try deliberate shortcuts: hard-coded outputs, skipped controls, cited but unsupported claims. Confirm the grader catches them.
- Check data provenance for every task source, including licenses and any client-owned data.
- Confirm how the vendor retires saturated tasks and patches graders after a hack is found.
- Ask whether the tasks are exclusive to you and what that does to price.
If expert scores and environment rewards disagree often, the grader is the problem. Fix it before training, not after.
What should you ask an RL environment vendor?
- Who wrote these tasks, and how did you verify their credentials?
- Which tasks use verifiable checks, which use rubrics, and which use an LLM judge?
- How do you score intermediate steps when the final answer needs a wet lab?
- What reward hacks have you found, and how did you close them?
- What is the pass-rate distribution on a current frontier model?
- How often do experts review graded rollouts, and what is the agreement rate?
- Where did the scientific data come from, and who owns it?
- Can we audit any rollout, grader change or task revision?
What do RL environments cost?
Prices vary widely, so ask for a per-task quote. Epoch AI reports environment contracts often run six to seven figures per quarter, with typical task costs of $200 to $2,000 and $20,000 per task rare but possible. Two founders told Epoch that exclusive deals cost roughly four to five times more than non-exclusive ones. Epoch also cites The Information's report that Anthropic discussed spending over $1 billion on environments over a year.
The supply side is young. SemiAnalysis counts more than 35 environment companies, most of them seed-stage with fewer than 20 employees. Science domains add another cost: the experts who write tasks and audit rewards.
Should you build or buy?
Build the parts that use your proprietary data and define your edge. Buy or co-build the parts that need many credentialed experts at once: task authoring, rubric design, and blind grading of rollouts. Epoch reports labs are bringing substantially more environment work in-house, while still using vendors for scale. Most life sciences teams will do both.
The Isomorphic–MSD collaboration shows the logic. One side brings models, the other brings data that nobody else has. Your environment strategy should protect the second.
How Acumen helps
We supply the expert layer that makes life sciences environments sound. Our verified PhD scientists, pharmacologists and bioinformaticians write tasks, build rubrics for intermediate steps, grade rollouts blind, and red-team graders for reward hacks. Every expert is credential-checked, and every grade is auditable. See our life sciences model evaluation capability, or read how we approach rubric-based RL for medical AI.
FAQ
What is an RL environment for life sciences AI?
It is a sandbox where a model attempts a scientific task, such as analyzing sequencing data or planning a cloning protocol, using tools and data, then receives a reward from a grader. The task, the tools and the grader together decide what the model learns. In science, the grader is usually the weakest part and needs expert design.
Why is reward hacking a bigger risk in science?
Scientific answers are often hard to check automatically. Final results may need a wet-lab experiment, and the literature can disagree with itself. That leaves room for answers that score well but are wrong. Builders interviewed by Epoch AI ranked robustness to reward hacking as the top quality criterion for any environment.
How do you score biology tasks that need a wet lab?
Score the steps you can check. SemiAnalysis suggests rewarding intermediate steps, approximated with rubrics, and splitting long tasks into individually graded pieces. In practice, scientists write rubrics for design choices, controls and analysis, while computational checks cover code and data. Confirm a sample of rubric grades with independent experts.
What pass rate should RL environment tasks have?
Not 0% and not 100%. Environment builders told Epoch AI a useful floor is about 2–3%, or one success in 64–128 rollouts, so the model gets some signal. They suggested a spread of difficulty and retiring tasks once they pass around 70% of the time.
How much do RL environments cost?
Epoch AI reports typical task costs of $200 to $2,000, with $20,000 per task rare but possible, and contracts often six to seven figures per quarter. Exclusive deals cost roughly four to five times more. Life sciences tasks add the cost of credentialed experts to write tasks and audit rewards.
Who should write and grade life sciences RL tasks?
Working scientists in the task's domain. One environment founder told Epoch AI that domain knowledge matters more than ML skills for writing tasks. For biology and chemistry, that means verified PhDs who write the tasks and rubrics, plus a separate expert panel that grades rollouts blind to check the reward.