Healthcare · · 8 min read
What the HHS clinical AI roadmap means for health systems
Acumen Editorial · Updated November 20, 2026
On November 13, 2026, HHS published "A Clinical Super Intelligence Roadmap: Better Answers, Faster Care, and Lower Costs." The eight-page document, hosted by CMS, is organized around four priorities: earning public trust, building the data foundation, setting clear rules and paying for what works.
Most coverage has focused on speed. We read it differently. Every pathway in the roadmap stays open only while evidence supports it. That makes evidence the product, and evidence comes from evaluation.
If you lead clinical AI at a health system, or you sell clinical AI to one, here is what the roadmap asks for and what to build now.
What does the HHS clinical AI roadmap actually say?
The core idea is a "better than today" bar. Clinical AI does not have to be perfect. It has to beat the care patients get now, for its risk tier. According to a summary by CASRAI, HHS says it will not wait for flawless evidence to open pathways, but will keep them open only if the evidence holds.
The roadmap pairs that bar with five mechanisms:
- Staged deployment. Tools earn trust and autonomy step by step. Nothing gets more autonomy than its evidence supports.
- Risk-aligned FDA review. Low-risk tools, including clinical decision support software, keep a fast path. Higher-risk tools could face a competency-based review, which the roadmap compares to how physicians are credentialed.
- Real-world surveillance. FDA and CMS would work together on surveillance infrastructure, alongside predetermined change control plans.
- State sandboxes. HHS proposes partnering with states on testbeds. Nextgov notes Utah is already piloting AI-issued prescriptions and refills.
- Outcomes-based payment. CMS anticipates extending the ACCESS model to primary care and exploring outcomes-based payment in Original Medicare and Medicaid.
The roadmap also sets a data baseline. It says the CMS Health Tech Ecosystem grew from more than 60 company commitments to over 800, and patient digital access to records rose from roughly 5% to 60% in a year.
What does "better than today" mean for evaluation?
It means you need two measurements, not one. You need the model's performance on real clinical tasks, and you need a baseline for current care on the same tasks. A score with no comparison cannot show that a tool beats today.
That baseline should come from practicing clinicians working the same cases under realistic conditions. It should be graded with the same rubric as the model. And it should be broken down by task and risk, because "better on average" can hide a tool that is worse on the cases that matter most.
It also means worst-case results count. A tool that improves routine answers but adds rare severe errors does not clear a safety bar, whatever its mean score.
How does FDA's competency-based approach fit in?
The roadmap points to a direction FDA already floated. On August 18, 2026, FDA's device center released a discussion paper on generative AI-enabled devices. Cooley summarizes the proposal in two parts.
Device benchmarking is a scalable, nonclinical test of whether the deployed device shows adequate clinical knowledge, analytic capability, safety behavior, communication and generalizability. Agentic functions get their own competencies.
Clinical confirmation checks that benchmark performance holds in real use. Evidence can range from retrospective analysis of real patient data to prospective trials, scaled to risk.
The paper is not guidance, and comments closed October 19. But the shape is clear. A competency-based review needs a competency-based test set: cases written and graded by clinicians, organized by skill, and kept out of training.
What will post-market surveillance require?
The FDA paper describes three post-market approaches, each scaled to risk:
| Approach | What it means | What you need to run it |
|---|---|---|
| Periodic benchmarking | Rerun the device against the same thresholds on a schedule and after model changes | A frozen, held-out evaluation set and documented pass thresholds |
| Sample-based clinician review | Qualified, independent clinicians review sampled real-world inputs and outputs against defined criteria | A pool of credentialed specialist reviewers, written criteria and adjudication rules |
| Degradation monitoring | Detect drift from changes in patient mix, data or the underlying model | Baseline metrics by subgroup and a trigger for re-review |
The roadmap adds FDA's TEMPO pilot, in which manufacturers can request enforcement discretion in exchange for sharing real-world performance data. Either way, the direction is the same. Evaluation does not end at launch.
One gap matters for buyers. Cooley notes the FDA paper does not say how independent clinician adjudicators would be credentialed or contracted. Health systems and vendors will need to answer that themselves, and document it.
What should health systems do now?
Build the evidence before anyone asks for it. Here is the checklist we use.
- Inventory every clinical AI tool and assign each a risk tier.
- For each tool, define the clinical tasks it performs and the outcome that counts as success.
- Build a private evaluation set of clinician-written cases with rubrics, per tier and specialty.
- Measure current-care performance on the same cases to set the "better than today" baseline.
- Report worst-case and subgroup results, not just averages.
- Rerun the frozen set on every model or prompt change and compare case by case.
- Stand up sample-based review: a set share of live outputs graded by independent specialists each month.
- Document reviewer credentials, criteria, inter-rater agreement and adjudication.
- Ask every vendor for the same evidence, and for rights to rerun your set on their updates.
This also lines up with HHS's own priorities. In June 2026, HHS named "clearer standards for evaluating AI tools" as one of three priorities drawn from more than 7,000 comments on its request for information, according to GovCIO Media.
What should you ask clinical AI vendors?
The roadmap shifts the burden toward evidence, so ask for it directly.
- Which risk tier do you place this tool in, and why?
- What is your baseline, and how was current care measured?
- Who wrote and graded your evaluation cases, and in which specialties?
- How do you monitor performance after deployment, and how often do clinicians review live outputs?
- What happens to performance when the underlying model changes?
- If outcomes-based payment applies, what outcome data can you share?
Large developers are publishing their methods. On November 4, 2026, Google described a health AI evaluation framework covering accuracy, safety and usefulness, combining benchmarks, automated graders and expert human review. Ask your vendors whether theirs is as explicit.
How Acumen helps
We build the evaluation layer the roadmap assumes. Practicing physicians, matched by specialty, write competency-organized cases and rubrics. They also work the same cases to set the current-care baseline. Specialists grade model outputs, a third reviewer adjudicates disagreements, and we document reviewer credentials, agreement and test-data independence for procurement and regulatory review.
After launch, we run sample-based clinician review of live outputs and rerun your frozen set on every model change. We work only in healthcare, life sciences and pharma, so the people grading your model know the specialty.
See how it works on our Medical Model Evaluation page. For the method behind it, read How to evaluate a clinical LLM. For adversarial testing before staged rollout, see Healthcare AI Red Teaming.
FAQ
What is the HHS clinical AI roadmap?
It is an eight-page HHS document published November 13, 2026, titled "A Clinical Super Intelligence Roadmap: Better Answers, Faster Care, and Lower Costs." It is organized around four priorities: earning public trust, building the data foundation, setting clear rules and paying for what works. It proposes a "better than today" safety bar, staged deployment, real-world surveillance and outcomes-based payment.
What does "better than today" mean for clinical AI?
It means a tool must outperform the care patients currently receive for its risk tier, not achieve perfection. In practice, that requires measuring the model and current clinical practice on the same tasks with the same rubric. Results should be broken down by task, subgroup and severity, so a better average does not hide worse performance on high-risk cases.
What is FDA's competency-based evaluation for generative AI devices?
It is an approach floated in an FDA device center discussion paper on August 18, 2026. It has two parts: device benchmarking, a nonclinical test of clinical knowledge, safety behavior, communication and generalizability; and clinical confirmation, which checks real-world performance with evidence scaled to risk. The paper is not guidance, but the roadmap points in the same direction.
What post-market monitoring will clinical AI need?
FDA's discussion paper describes periodic benchmarking against the same premarket thresholds, sample-based review of real-world outputs by qualified independent clinicians, and monitoring for performance degradation. The roadmap adds FDA and CMS collaboration on surveillance infrastructure and the TEMPO pilot, which trades enforcement discretion for shared real-world performance data. Evaluation becomes an ongoing program, not a launch gate.
Does the roadmap change how clinical AI gets paid for?
It signals a shift toward outcomes. CMS anticipates extending the ACCESS model to primary care, building structures for AI-led solutions in Innovation Center models and exploring outcomes-based payment in Original Medicare and Medicaid. AI-led solutions would need clinician partnerships so patients can escalate to a human. Outcome evidence becomes part of the business case.
What should a health system do first?
Inventory its clinical AI tools and assign each a risk tier. Then build a private, clinician-written evaluation set for each tool, measure current care on the same cases to set a baseline, and rerun the set on every model change. Stand up sample-based specialist review of live outputs and document reviewer credentials and agreement.