Clinical AI evaluation imagery representing expert review of healthcare workflows.

Clinical AI evaluation · · 7 min read

How to evaluate AI agents in the EHR

Acumen Editorial · Updated November 21, 2026

AI agents are moving from the chat window into the chart. They read records, draft notes, place orders and work payer portals. Vendors now sell agents that act, not just answer. That changes what an evaluation has to prove.

Stanford's latest benchmarks show the gap. On PhysicianBench, 100 long-horizon tasks adapted from real primary care to specialist consults across 21 specialties, the best model completed 46 percent of tasks on one attempt. On HealthAdminBench, 135 prior authorization, appeal and equipment-order tasks, the best agent finished 36.3 percent end to end. Stanford HAI summed it up in October 2026: "Today's best AI agents complete less than half the work."

Those are public tests. Your workflows, order sets and payers are not in them. Here is how we would test an agent before it touches your EHR.

Why do exam scores fail for EHR agents?

Because an agent fails in ways an exam cannot see. Multiple-choice benchmarks grade a single answer. An agent has to find the right data, decide, then act through tools, often dozens of times in a row. PhysicianBench tasks average about 27 tool calls each.

The parts can look strong while the whole fails. On HealthAdminBench, GPT-5.4 passed 82.8 percent of subtasks but completed only 26.7 percent of whole tasks, according to Stanford HAI. As the post puts it, "Competence on the parts does not add up to reliability in a whole workflow."

The most telling failure is quiet. Among action failures on PhysicianBench, 62 percent for GPT-5.5 and 73 percent for Claude Opus 4.6 involved writing the right action in a note without placing the order. The note looks correct. The patient gets nothing.

What should an EHR agent evaluation cover?

Every step the agent takes, and the state it leaves behind. We split the work into five layers:

  • Retrieval. Did the agent find the right labs, notes, medications and history? FHIR-AgentBench, presented at the 2026 Machine Learning for Health symposium, grounds 2,931 clinical questions in HL7 FHIR and finds that both retrieving from complex resources and reasoning over them hurt accuracy.
  • Clinical judgment. Was the plan right for this patient, by the standard a specialist would hold?
  • Action. Did the order, referral, message or form actually land in the system, with the right fields?
  • Documentation. Does the note match what was done, without invented findings?
  • Safety stops. Did the agent pause, escalate or refuse when it should have?

Microsoft Research's HealthAgentBench adds the data layer. Its 75 tasks span imaging, pathology and EHR data, and agents often missed subtle conflicting or duplicate records when auditing data. The best agent succeeded on 57.3 percent of tasks.

Public benchmarks vs. a private evaluation: which do you need?

Both, for different jobs. Public benchmarks tell you which models are worth shortlisting. A private evaluation tells you whether an agent works in your system.

QuestionPublic benchmarkPrivate evaluation on your workflows
Which models are capable at all?Good fitOverkill
Does it work with our EHR build and order sets?NoYes
Does it handle our payers and forms?NoYes
Are results contaminated by training data?Risk grows over timeLow, if the set stays private
Who wrote the reference standard?Benchmark authorsYour clinicians and administrators
Can we rerun it after every update?Fixed tasksYes, as a regression suite

How should you build EHR agent test tasks?

From your own work, in a sandbox. Here is the checklist we use:

  • Pick the workflows. Name the exact jobs the agent will do: consult prep, order entry, inbox replies, prior authorization, appeals.
  • Start from real cases. Adapt de-identified consults, denials and orders, the way PhysicianBench adapted real specialist consults.
  • Have clinicians write the reference. For each task, the steps, the correct end state and what must never happen.
  • Break tasks into checkpoints. PhysicianBench uses 670 checkpoints across 100 tasks. Checkpoints show where an agent fails, not just that it failed.
  • Verify against the system, not the transcript. Check what the agent actually changed in the sandbox EHR.
  • Seed traps. Conflicting records, outdated medications, a missing consent, a payer rule change.
  • Run each task several times. GPT-5.5 completed 46 percent of PhysicianBench tasks on one try but only 28 percent in all three runs.
  • Grade with the right people. Physicians for clinical judgment, pharmacists for medication tasks, revenue cycle staff for payer work.
  • Keep the set private and rerun it. New model, new EHR upgrade or new payer rule means a new run.

Who should grade an EHR agent?

People who do the work today. HealthAdminBench was built from more than 100 hours of watching administrators at Stanford Health Care. That is why its tasks feel real.

Deterministic checks cover a lot: was the order placed, was the field filled. HealthAdminBench uses 1,177 of them. But it also needed 521 rubric checks for judgment calls. Those rubrics need expert authors, and any model grader needs to be checked against clinicians before you trust it.

Which metrics matter?

Report the numbers a clinical leader would act on:

1. End-to-end completion rate, by workflow.

2. Consistency, the share of tasks completed on every run.

3. Silent failures, where the agent reports success but the system state is wrong.

4. Unsafe actions, with severity, reported on their own.

5. Escalation quality, whether the agent stopped at the right moments.

6. Time and tool calls per task, so you can compare cost.

Average accuracy hides the misses that matter. One wrong order outweighs a hundred correct summaries.

What should you ask an EHR agent vendor?

1. Which workflows was the agent tested on, and in whose EHR build?

2. Who wrote the reference end states, and what are their credentials?

3. What is end-to-end completion, and how consistent is it across repeated runs?

4. How do you detect silent failures, like a note that says an order was placed when it was not?

5. Which actions require human approval, and can we change that?

6. Can we run our own private task set before go-live and after every update?

How Acumen helps

We build the private evaluation sets that health systems and AI teams use to test agents before they go live. Our medical model evaluation service supplies workflow tasks written by practicing clinicians and administrators, checkpoint-level reference end states, seeded traps and blind graded runs, matched by specialty. Every task is reviewed and adjudicated.

Use it to compare vendors, to tune your own agent, or as a regression suite after each release. For testing the model behind the agent, read our guide on how to evaluate a clinical LLM.

Request a sample dataset Schedule a conversation

FAQ

What is an EHR AI agent?

An EHR AI agent is a model that takes actions inside the health record, not just answers questions. It can retrieve data, draft notes, place orders, send messages or work payer portals through tools and APIs. Because it changes the system, it must be judged on the end state it leaves, not only on what it says.

How accurate are AI agents in the EHR today?

Not accurate enough to run unsupervised. On Stanford's PhysicianBench, the best model completed 46 percent of 100 specialist consult tasks on one attempt. On HealthAdminBench, the best agent completed 36.3 percent of administrative tasks end to end. Results on your own workflows can differ, which is why private testing matters.

Why can't we rely on public healthcare agent benchmarks?

Public benchmarks are useful for shortlisting, but they do not use your EHR build, order sets, payers or forms. Their tasks can also leak into training data over time. A private task set, written by your clinicians and kept out of vendor hands, tells you how an agent performs in your environment and after every update.

What is a silent failure in an EHR agent?

It is when the agent appears to succeed but the system state is wrong. Stanford found that most action failures on PhysicianBench involved writing the correct action in a note without placing the order. Catch these by verifying what changed in the sandbox EHR, not by reading the agent's own summary.

How many times should each task be run?

At least three. Agents are not deterministic, so one run overstates reliability. On PhysicianBench, GPT-5.5 completed 46 percent of tasks on a single attempt but only 28 percent in all three runs, per Stanford HAI. Report both single-run completion and consistency across runs before any go-live decision.

Who should write and grade EHR agent tasks?

The people who do the work. Physicians should write and grade clinical tasks, pharmacists medication tasks, and revenue cycle staff prior authorization and appeals. Use deterministic checks where possible, expert-written rubrics for judgment calls, and validate any model grader against clinician scores before relying on it.

From the Acumen archive. Explore expert data and Healthcare & Pharma.

← Back to Insights