Pharma · · 8 min read
Pharma AI agent governance: how to test agents before they reach the field
Acumen Editorial · Updated November 21, 2026
Pharma has moved fast on AI. Governance has not kept up. That is the headline of the Responsible AI in Life Sciences Report 2026 from ACTO, covered by PharmaVoice on November 17, 2026. ACTO surveyed 135 life sciences leaders and ran eight executive roundtables with 80 participants across North America and Europe.
The numbers are plain. More than one-third of companies have no defined internal AI governance or rely on one-time IT approvals. 23% say their organizations do not monitor or discuss employee use of unauthorized AI agents at all. 58% believe the industry still lacks clear standards for responsible AI use.
This matters most where agents now work: drafting clinical summaries, answering medical information questions, triaging literature, preparing slides for field teams. Those outputs reach physicians. When they are wrong, someone has to answer for it.
Why is pharma AI agent governance lagging adoption?
Because building an agent got easy before checking one did. PharmaVoice reports that some companies hold enterprise-wide licenses that let any employee build agents. The result is "shadow AI": agents in daily use whose output nobody measures.
Internal LLM platforms are now standard at the top 50 pharma companies. Consilience Strategy's 2026 analysis notes medical affairs has been slower to adopt AI because of its regulatory and accuracy demands. Those two facts collide when a self-built agent starts drafting medical content outside any review process.
Most governance today was designed for software purchases. It asks whether a tool is secure and approved. It rarely asks whether this agent, on this task, gives answers an expert would sign.
Who is accountable when an agent gets it wrong?
Pharma leaders do not agree. PharmaVoice found proposed owners ranging from the end user to legal and compliance, vendors and IT.
The article describes the risk in concrete terms: a medical science liaison presenting inaccurate data after an agent drafted clinical summaries, and a chemistry agent reaching a wrong conclusion from flawed data. In both cases the error passed through a person who trusted the output.
ACTO's CEO Parth Khanna put the industry view simply: "only humans can be held accountable." We agree, and it has a practical consequence. If a person is accountable for an agent, that person needs evidence the agent works on the tasks it performs. A vendor demo or a general benchmark is not that evidence.
We recommend a simple split:
- Business owner: the function head whose team uses the agent. Owns the context of use and signs off on launch.
- Expert reviewers: MSLs, medical information specialists, medical writers or scientists who build and grade the test set.
- Quality or compliance: confirms the test record meets internal standards and decides how often to re-test.
- IT and security: owns access, data handling and model change logs.
One-time approval vs. lifecycle governance: what changes?
The FDA-EMA Guiding Principles of Good AI Practice in Drug Development, published January 14, 2026, name ten principles. Four of them describe what one-time approval misses: clear context of use, multidisciplinary expertise, risk-based performance assessment and life cycle management.
| Question | One-time IT approval | Lifecycle governance |
|---|---|---|
| What is approved? | The tool or platform | Each agent, for a written context of use |
| Who reviews it? | IT and security | Business owner, domain experts, quality, IT |
| What evidence is required? | Security review, vendor documentation | Expert-graded test results on your own tasks |
| When is it re-checked? | Rarely, at contract renewal | After any model, prompt, tool or source change, and on a schedule |
| How are live outputs checked? | Not at all | A sampled share of outputs graded by experts |
| What happens on failure? | Unclear | Defined rollback, owner notified, incident logged |
| Who answers for errors? | Disputed | A named business owner, backed by a test record |
ACTO's framework points the same way. PharmaVoice lists the areas institutional guidelines should cover: governance, security, testing and certification, human centricity, value creation and orchestration across agents, systems and data. Testing and certification is the part most teams have not built.
How do you test an agent that drafts clinical content?
Start with the work, not the model. Pull a set of real tasks the agent will face: the summaries, answers or slides your team produced last quarter. Ask the experts who did that work to write reference answers and grading criteria before the agent runs.
Then grade what matters to the person who will be accountable:
- Factual accuracy: every number, endpoint and population matches the source.
- Completeness: safety information, limitations and comparators are not dropped.
- Source fidelity: every claim traces to the document it cites.
- Scope: off-label or out-of-scope requests are recognized and routed.
- Consistency: the same task, run several times, gives the same answer.
Size the set to the risk. An internal drafting aid can launch on a modest set of tasks. An agent whose output reaches physicians needs coverage of every product, indication and question type it will see, including the hard cases: conflicting studies, missing data, safety signals and requests it should refuse.
Record what the agent got wrong, not only its pass rate. A failure log tells the accountable owner where the agent should never be used without a human check, and it gives quality teams something concrete to review.
Grade blind, with at least two reviewers on high-risk items and a documented way to settle disagreements. Keep the test set private so it does not leak into vendor training data. Re-run it whenever the underlying model, prompt or retrieval sources change.
Writing in PharmExec on November 16, 2026, Dan Barton argued that expert review belongs inside the work, not only at the end: "Judgement matters most when the stakes and ambiguity are highest." That is exactly where agent testing should concentrate.
Checklist: governance for every pharma AI agent
- The agent has a named business owner who signs off on launch.
- Its context of use is written down: task, users, therapeutic area, decisions it informs.
- It is registered in a central inventory, including agents built by employees.
- A private test set was built from real tasks by experts who do that work.
- Grading criteria were set by those experts before testing began.
- High-risk items are graded by at least two experts, with disagreements logged.
- Pass thresholds are documented and proportionate to risk.
- Every model, prompt, tool or source change triggers a re-test.
- A sample of live outputs is graded by experts on a set schedule.
- Failures trigger rollback, owner notification and an incident record.
- Reviewer credentials, item provenance and test versions are documented.
- Quality or compliance has reviewed the test record once before launch.
How Acumen helps
We build the expert test sets that make agent governance real. Verified MSLs, medical information specialists, medical writers and regulatory experts write tasks from real work, set grading criteria before the agent runs and grade outputs blind with multi-review adjudication. Each item ships with reviewer credentials, provenance and version history, so the accountable owner has a record to stand behind.
See medical affairs and regulatory AI data. For how regulators are framing AI evidence, read EMA AI guidance: what pharma data teams should do now.
FAQ
What is pharma AI agent governance?
It is the set of rules, owners and tests that decide whether an AI agent can be used for a given task and how it is checked over time. Good governance names an accountable business owner, defines the context of use, requires expert-graded testing before launch and re-tests the agent whenever its model, prompt or sources change.
Who should be accountable for AI agent errors in pharma?
A named business owner from the function that uses the agent, such as the head of medical information for a medical information agent. ACTO's 2026 research found leaders disagree, naming end users, legal, vendors and IT. Accountability works only when that owner has expert test results showing the agent performs on the tasks it handles.
What is shadow AI in pharma?
Shadow AI is AI use that governance cannot see, such as agents employees build on enterprise licenses without registration or review. ACTO's 2026 survey found 23% of life sciences organizations do not monitor or discuss employee use of unauthorized AI agents at all. An agent inventory that includes self-built agents is the first fix.
Is one-time IT approval enough for AI agents?
No. IT approval confirms a platform is secure, not that a specific agent gives correct answers. Agents change when models, prompts or sources change. The FDA-EMA principles call for risk-based performance assessment and life cycle management, which means testing each agent for its context of use and re-testing it over time.
How should pharma test an agent that drafts clinical summaries?
Build a private test set from real summaries your team has produced. Have MSLs or medical writers write reference answers and grading criteria before testing. Grade accuracy, completeness, source fidelity, scope handling and consistency across repeated runs. Use at least two blind reviewers on high-risk items and re-test after every model or prompt change.
How often should AI agents be re-tested?
Re-test after any change to the underlying model, prompt, tools or retrieval sources, and on a fixed schedule even when nothing changes. Between full re-tests, have experts grade a sample of live outputs. Set the frequency by risk: agents whose output reaches physicians need tighter cycles than internal drafting aids.