Healthcare · · 7 min read
Expert AI training data cost: what drives it and how to budget
Acumen Editorial · Updated November 20, 2026
Expert data is now one of the biggest line items in post-training. On November 10, 2026, Mercor announced a $750 million Series D at a $20 billion valuation and said more than five million experts in its network are paid to train frontier models. Sacra estimates Mercor reached about $3 billion in annualized gross revenue at the end of October 2026, up from $760 million at the end of 2025.
That growth is money buyers are spending. And most of it goes to experts. Sacra reports that contractor payouts make up 60–70% of Mercor's gross revenue. So when you price expert data, you are mostly pricing expert time. The question is how much of that time ends up in a usable dataset.
This guide breaks down what drives the price, how vendors charge, and how to budget a healthcare, life sciences or pharma data project without paying for items you can't use.
Why does expert data cost so much more than labeling?
Because the work changed. The market has moved from cheap generic labeling to expensive expert review, and reasoning models created demand for doctors, lawyers and PhDs (Sacra). A crowd worker can draw a bounding box. Only a nephrologist can tell you whether a model's dosing advice is safe for a patient on dialysis.
The rates reflect that. Fast Company reported in July 2026 that Mercor pays experts an average of $80 per hour. Bloomberg Law reported in October 2026 that a radiologist earns in an hour what a general data worker earned in a week, and its headline put top rates for doctors and lawyers at up to $400 an hour.
The tasks are also longer. Physicians now write full cases from their own practice, build rubrics, grade clinical reasoning and compare model responses side by side. OpenAI's HealthBench used 262 physicians to write 48,562 rubric criteria. That is not a quick-click job, and it should not be priced like one.
What actually drives the price?
Five variables set almost every quote. Ask each vendor to state them for your project.
| Cost driver | What it means | Why it moves the budget |
|---|---|---|
| Expert rate | Hourly pay for the specialty and seniority you need | Subspecialists cost more and are scarcer than generalists |
| Minutes per item | Time to write, grade or compare one item | A full case can take many times longer than a preference judgment |
| Review layers | How many independent experts check each item | Two or three reviewers can double or triple expert hours |
| Acceptance rate | Share of items that pass QA and ship | Every rejected item is paid for and thrown away |
| Vendor margin | Recruiting, QA, tooling, project management and profit | The share above expert payouts; ask what it covers |
The first two are what most teams negotiate. The last three are where the real cost hides.
Per hour, per task or per accepted item: which pricing model is best?
Vendors price expert work in three main ways. Each shifts risk differently.
| Pricing model | You pay for | Risk sits with | Best for |
|---|---|---|---|
| Per expert hour | Time worked, plus a markup | You: slow or weak work still bills | Early pilots and exploratory rubric design |
| Per task | Each item submitted | Shared: you still pay for items that fail QA unless the contract says otherwise | Stable, well-defined task types at volume |
| Per accepted item | Only items that pass agreed QA | The vendor: rework is its cost | Production datasets with clear acceptance criteria |
We recommend per-accepted-item pricing once a task type is stable. It forces both sides to define "good" before the work starts, and it turns rework into the vendor's problem. For a first pilot, hourly pricing is fine. Just measure minutes per item and acceptance rate so you can convert to a per-item price later.
How much review do you really need?
Enough to match the risk of the deployment. Review layers are the biggest multiplier in a healthcare budget, and the hardest to cut safely.
The benchmarks show why. HealthBench Professional adjudicates its rubrics with three or more physicians. The Stanford- and Harvard-led NOHARM study found omissions made up more than 80% of severe errors, and omissions are exactly what a single reviewer misses. A 2025 survey of HealthBench results warned that strong averages can hide dangerous weaknesses, so worst-case performance matters more than the mean.
A practical rule: one expert plus spot checks for low-risk preference data, two blind reviewers with a written tie-break rule for clinical reasoning data, and three or more with senior adjudication for evaluation sets you will report to a regulator or a board.
Where do expert data budgets leak?
The waste rarely comes from the hourly rate. It comes from four places.
Wrong specialty match. A general internist grading oncology regimens produces items that fail review or, worse, pass it. Paying a lower rate for the wrong expert is the most expensive choice on the table.
Vague guidelines. If experts don't know what a good answer looks like, acceptance rates fall and every item gets reviewed twice. A short pilot that fixes the guideline before scale pays for itself.
Unmeasured rework. Hourly contracts hide rejected work inside total hours. Ask for acceptance rate and rework hours per batch.
Missing documentation. FDA's January 2025 draft guidance on AI-enabled device software asks sponsors to describe how training data was collected, cleaned and processed. Rebuilding provenance records after the fact costs more than capturing them during the project.
Checklist: what to ask a vendor about pricing
- Which pricing model do you offer: hourly, per task or per accepted item?
- What is the expert rate by specialty, and what share of my price reaches the expert?
- How many minutes per item did comparable projects take?
- How many reviewers check each item, and is that included in the price?
- What acceptance rate did your last comparable project hit?
- Who pays for items that fail QA?
- What does your margin cover: recruiting, QA, tooling, project management?
- Is provenance and review documentation included, or extra?
- What does a pilot cost, and how does it convert to a production price?
- Can I see a sample dataset before I commit to volume?
How Acumen helps
We work only in healthcare, life sciences and pharma. Acumen matches verified clinicians and scientists by specialty, runs multi-expert review with documented adjudication, and agrees written acceptance criteria with you before production starts. You see the specialty, review depth and acceptance criteria before the first batch ships. Every dataset includes the provenance records procurement and regulatory teams ask for.
See expert data for AI labs entering healthcare. For physician preference pairs and reward data, see RLHF data for healthcare. For sourcing criteria beyond price, read how to source expert-verified healthcare AI training data.
FAQ
How much does expert AI training data cost?
It depends on specialty, task length, review depth and acceptance rate. Fast Company reported Mercor pays experts an average of $80 per hour, and Bloomberg Law reported top rates for doctors and lawyers of up to $400 an hour. The useful number is cost per accepted item, which includes review time and rejected work, not the hourly rate alone.
Why is physician data more expensive than general annotation?
Physician tasks require clinical judgment that crowd workers do not have: writing realistic cases, building rubrics, grading reasoning and spotting unsafe omissions. They take longer per item, need scarce specialists and usually need more than one reviewer. Sacra describes the market as moving from cheap generic labeling to expensive expert review for exactly this reason.
Is it better to pay per hour or per accepted item?
Pay per hour for early pilots, when the task and guidelines are still changing. Move to per-accepted-item pricing once the task is stable and acceptance criteria are written down. That model makes rework the vendor's cost and gives you a predictable price per usable item, which is easier to defend in procurement.
How many expert reviewers does healthcare AI data need?
Match review depth to risk. Low-risk preference data can use one expert with spot checks. Clinical reasoning data should have two blind reviewers and a written tie-break rule. Evaluation sets you will report externally deserve three or more, the standard HealthBench Professional uses for its physician-adjudicated rubrics.
What share of an expert data price goes to the experts?
It varies by vendor and contract. Sacra reports that contractor payouts account for 60–70% of Mercor's gross revenue, with the rest covering operations and profit. Ask any vendor what share of your price reaches the expert and what the remainder covers, such as recruiting, QA, tooling and project management.
How do I keep an expert data budget under control?
Run a short pilot to fix guidelines and measure minutes per item and acceptance rate. Match experts by specialty rather than by lowest rate. Agree acceptance criteria in writing, track rework per batch and capture provenance documentation during the project. Most overruns come from rejected items and rework, not from the hourly rate.