OpenAI Medical Agent Can Now Operate Hospital Systems: 3 Sets of Evaluation Data Show How AI Moves from Answering Questions to Actually Doing Work

OpenAI Medical Agent Can Now Operate Hospital Systems: 3 Sets of Evaluation Data Show How AI Moves from Answering Questions to Actually Doing Work

openai-healthcare-computer-use-agent-hospital-systems-en-image-0

OpenAI’s medical computer-use agent marks a real step forward: it can get into the software systems healthcare organizations already use, read a patient’s longitudinal medical history, and use that information to prepare chart notes, order drafts, patient replies, and insurance prior-authorization paperwork. AI’s value is stretching from answering expert questions to executing actual business processes. The competitive focus in healthcare AI is shifting along with it — no longer confined to medical knowledge Q&A, but moving into clinical and administrative workflows themselves. One point worth stressing: in demos and public cases alike, every one of these materials gets reviewed by a physician or staff member. The AI doesn’t independently hold prescribing authority or medical decision-making power.

What you’ll take away from this article

  • The full chain of operations in the MRI demo, from reading the chart to preparing a prior-authorization request.
  • What each of the three figures actually measures — 99% safety, 93% strong physician approval, and 94.3% rubric points — and how they differ from the results reported in Stanford’s HealthAdminBench paper.
  • Where CVS Health, Cigna, Abridge, and Ambience each stand: synthetic demo, production deployment, or still in development.
  • How clinical approval authority and agent execution authority stay separated, and how developers should account for the full cost of an agent.

Index

The factual section unfolds in this order: first, what exactly OpenAI had the AI do this time; then, how a chatbot differs from a computer-use agent; next, what the benchmark scores actually mean; after that, how CVS, Cigna, Abridge, and Ambience are putting these tools to work; and finally, the risk controls built around medical agents. The analysis section looks at why healthcare in particular needs this capability, how developers and health systems should weigh the trade-offs, and what computer use means for the AI application industry as a whole.


What to Know: Medical Agent Concepts at a Glance

Concept Brief Explanation
Computer Use The ability to complete tasks by observing and operating software interfaces (screenshots, clicks, typing)
EHR Electronic Health Record system — the core software hospitals use to store patient visits, medications, imaging, and more
Payer Portal The insurance payer’s portal, where healthcare organizations submit prior authorizations and verify insurance eligibility
Prior Authorization Some tests or treatments need insurance approval before they can be performed
MCP A standard protocol for discovering and calling external tools
RPA Automation that runs repetitive processes through pre-configured rules or scripts
Rubric points Scoring points broken out by evaluation criteria, measuring how well partial requirements are met
End-to-end success rate The share of complete tasks that satisfy all success conditions
Trace Trajectory evidence that records the execution process, actions taken, and interface states
BAA Business Associate Agreement, which spells out a service provider’s responsibility for handling protected health information (PHI)
RBAC Role-Based Access Control, which assigns data and operations by role

Medical Agent Impact at a Glance

  • Developers and technical leads: The question is no longer whether the model can answer medical questions correctly. It’s whether you’ve validated the execution mechanism, end-to-end completion rate, and full cost for specific tasks. Screenshots count as image input, and multi-turn calls, tools, and runtime environments all get billed — so you can’t just look at the text token price.
  • Healthcare organizations and workflow leads: You need to separate synthetic-data demos, vendor evaluations, enterprise pilots, and production cases. When going live, you also need human approval, permission configuration, and audit logs in place — you can’t treat demo results as a procurement decision.
  • Clinical staff: In the short term, what they’ll most likely notice is a lighter administrative load — things like preparing prior authorization materials and tracking down information across systems. Clinical judgment and signing responsibility still stay with the doctors.

The Facts: What OpenAI’s Healthcare Agent Released This Time

What Did OpenAI’s Healthcare Agent Have the AI Do This Time?

On November 16, 2026, OpenAI announced new progress on a computer-use agent for healthcare teams. It also shared evaluation results for clinical and administrative workflows, along with partnership work involving CVS Health, Abridge, Ambience, Cigna Healthcare, and others. During HLTH 2026, Karan Singhal, who leads OpenAI Health AI, also discussed the progress.

The clearest illustration of this shift is the MRI workflow OpenAI showed. A patient messages a doctor about lower back pain, and the doctor considers ordering an MRI. The agent then completes a three-stage process across the EHR and the insurance portal:

  1. EHR information reading: reviews the patient’s visit records, physical therapy records, and prior imaging.
  2. Clinical material preparation: drafts the MRI order, the visit note, and the patient reply, all for physician review.
  3. Insurance prior authorization: uses the same patient record, enters the insurance payer portal, prepares the MRI prior authorization request, and hands it off to prior authorization staff for review.

To be clear, this demo uses synthetic patient data, not production work with real patients. It shows the capability chain; it doesn’t mean any hospital is already handling real cases this way.

openai-healthcare-computer-use-agent-hospital-systems-en-image-1

OpenAI also showed a medication consultation case after a child’s fracture. A family member asks about ibuprofen dosage for a child. The agent reviews visit records and imaging, reads the weight and clinic-administered dose recorded in the chart, and drafts medication instructions, cast care instructions, and warning symptoms that require urgent care. It saves the medical record as unsigned and the reply to the family as an unsent draft, with final approval left to the doctor. Elevated PSA and Digoxin refill were also listed as task types in the EHR Inbox, but OpenAI did not publish complete operation logs or independent scores for those two task types.

From Chatbot to Computer-use Medical Agent: What’s Changed?

Traditional medical Q&A answers general knowledge, and medical document generation can write a draft—but the content still has to be copied into the system by a person. What makes a Computer-use Agent different is that it enters existing software directly through a loop of “screenshot observation → planning → click or type → screenshot to verify,” completes multi-step tasks across applications, and may change business state in the software.

Capability form Primary output Does it interact with business systems? Typical medical scenario Main risk point
Medical Q&A Chatbot Text answers No Medical knowledge queries, evidence retrieval Content accuracy
Medical document generation Notes, draft letters Indirectly; requires manual entry Discharge instructions, pre-authorization letters Content inconsistent with the medical record
Computer-use Agent Interface operations and system state changes Yes; directly operates EHRs and portals Pre-authorization form filling, referral status updates, cross-system data retrieval Misclicks, patient mix-ups, error propagation

In the OpenAI API, Computer Use is mainly implemented through the Responses API and supports actions such as click, type, scroll, keypress, drag, and screenshot. The official documentation describes two paths: first, Computer Tool, where the model returns structured mouse and keyboard actions for the application to execute; second, Code Execution, where the model generates code that calls tools such as Playwright and PyAutoGUI. For GPT-6 Astra, the official recommendation currently is the Code Execution approach. The documentation also specifically emphasizes that the model generating an operation instruction does not mean the operation has already succeeded—the application must execute the action and then confirm the result through a new screenshot or other means.

What Do OpenAI’s Medical Agent Evaluation Scores Actually Tell Us?

OpenAI published three sets of headline numbers. Each has a different denominator and measures a different thing, so they can’t be merged into a single “overall accuracy” — and none of them is a medical diagnostic accuracy rate.

openai-healthcare-computer-use-agent-hospital-systems-en-image-2

Metric Value Model evaluated Denominator / basis What it doesn’t mean
Clinical task safety 99% GPT-6 Astra 325 reviewed tasks; assesses whether an error, omission, or action could possibly harm a patient Diagnostic accuracy, zero errors
High physician endorsement 93% GPT-6 Astra 646 ratings; combined share of Very / Extremely Helpful on a five-point scale Task success rate, 646 individual physicians
HealthAdminBench 94.3% GPT-6 Astra Share of rubric points, maximum reasoning effort Full-task success rate
HealthAdminBench 91.8% GPT-6 Luna Share of rubric points, maximum reasoning effort Full-task success rate
Cost per task $0.085 GPT-6 Luna Specific evaluation conditions, 98% lower than Astra Production deployment pricing

The clinical review didn’t look only at the text the model produced. It also examined the actions the agent actually took, the medical records it generated, the orders it wrote, and the messages it sent to patients. In other words, the evaluation covers both content quality and the software workflow. That said, OpenAI didn’t disclose how many individual physicians the 646 ratings came from, and the announcement itself gives no confidence intervals, no per-task results, and no complete failure cases.

HealthAdminBench is a benchmark for healthcare administrative agents introduced by Stanford’s Shah Lab. According to the paper 2604.09937v1, the benchmark contains 135 tasks and 1698 subtasks, of which 1177 subtasks are scored deterministically and 521 by a model judge; in the paper, the best full-task success rate is Claude Opus 4.6 CUA at 36.3%, and the best subtask success rate is GPT-5.4 CUA at 82.8%. An earlier web version from Shah Lab had 70 tasks and 947 rubric points, and the best full-task success rate reported at the time was 22.9% — a useful historical reference point.

These numbers come from different versions, different models, different dates, different execution conditions, and different metrics, so they can’t be lined up into a single leaderboard. OpenAI’s 94.3% is a rubric-points figure, while the paper’s 36.3% is an end-to-end full-task success rate. The gap itself is a signal: for medical agents, “getting individual steps right” and “finishing the whole job” may be very far apart.

How Do CVS, Cigna, Abridge, and Ambience Put Medical Agents Into Practice?

The maturity of these enterprise case studies varies quite a bit. Below, I’ve grouped them by publicly disclosed status. These figures come mainly from disclosures by the companies and OpenAI; media reports serve as cross-validation, but that doesn’t make them equivalent to third-party audits.

Company Scenario Disclosure status Key facts
CVS Health Sending documents, referrals, and specialist consultations across clinical systems Production workflow Cleared a document backlog that had persisted for months in a single weekend; another agent handles tens of thousands of system operation steps related to consultations every day
Cigna Healthcare Cross-system information consolidation before care management Deployed Care managers previously had to navigate more than 10 systems; went from concept to deployment in under 8 weeks
Abridge Discharge barrier coordination In development Doctors use Abridge Assistant to request that an agent coordinate medication access and home care in parallel
Ambience Healthcare Insurance prior authorization applications Governance demo Pauses to ask the doctor when the medical record is ambiguous, and submits only after approval
Assort Closing the referral loop Official case study API writes the referral information, while Computer Use fills in status updates in the Athena interface

CVS’s “tens of thousands” refers to the volume of work involved in system operation steps—it doesn’t mean tens of thousands of clinical decisions were made independently by AI. CVS also open-sourced the CUA Agent Kit (Apache-2.0 license), which uses a two-stage architecture: first, a Computer-use Agent explores the workflow in natural language and records trace evidence such as screenshots, actions, and target locations; then a Codex Skill solidifies the successful trajectory into deterministic code, which is reviewed by engineers and then executed repeatedly. According to the README, the deterministic execution stage doesn’t require a model API key.

Assort’s case illustrates where Computer Use fits: Assort can already write referral information into the medical record through the Athena API, but the full loop still lacks one interface operation. The agent directly selects the referral doctor in the Athena interface, fills in the details, and updates the status to Visit Scheduled—this referral is then immediately removed from the staff’s follow-up list. The API handles structured operations, while Computer Use handles the parts the API doesn’t cover.

How Can Healthcare Agents Control Risk?

OpenAI’s published governance capabilities include RBAC, audit logs, application access scope configuration, human approval before critical actions, and controls that confine the Agent to specified workflows. Healthcare organizations can pause the Agent at any time and review step-by-step operation logs.

openai-healthcare-computer-use-agent-hospital-systems-en-image-3

This is essentially a separation-of-duties design: AI handles organizing information, preparing materials, and performing actions within its authorized scope; physicians are responsible for clinical judgment; prior authorization staff handle administrative review; and healthcare organizations configure permissions and oversee workflows. The Agent’s permissions must align with the boundaries of clinical responsibility.

On data compliance, OpenAI says organizational data in ChatGPT for Healthcare won’t be used to train foundation models, and CVS says its production data stays in its own infrastructure. But to support HIPAA-compliant workflows involving PHI, healthcare organizations need to use eligible services, ensure the services are covered by a BAA signed with OpenAI, configure security settings and data retention policies, and restrict the records and actions the Agent can access through organizational permissions. So “supporting HIPAA workflows” doesn’t mean “all OpenAI API calls are automatically compliant.”

As for availability, these capabilities are available through OpenAI API, ChatGPT for Healthcare, ChatGPT for Clinicians, ChatGPT Work, and Codex. Eligible U.S. clinicians can apply for ChatGPT for Clinicians for free; this free policy has geographic and eligibility restrictions. ChatGPT for Healthcare doesn’t have a publicly available, uniform fixed subscription price.


Analysis: Trade-offs and Industry Implications of Medical Agents

Why Does Healthcare Especially Need Computer Use-Style Agents?

Information fragmentation is especially pronounced in healthcare. A single visit can involve the EHR, imaging systems, prescription systems, insurance portals, referral systems, and discharge management platforms, often from different vendors with incomplete interfaces. MRI prior authorization is a perfect example: a doctor makes one clinical judgment, while staff have to repeatedly switch between medical records, physical therapy notes, imaging data, and the insurance portal to copy and upload information.

From this angle, Computer Use isn’t about whether AI understands medicine. It’s about whether information can move between systems. In my view, healthcare administrative automation is likely to reach commercial scale earlier than clinical diagnosis automation. The reason is that administrative tasks are high-volume, highly repetitive, clearly bounded, and verifiable through forms and system status, while liability risk is relatively contained. The Cigna case, which compressed information lookup across more than 10 systems into seconds, illustrates this “reduce the burden first, don’t replace people” path.

That said, the Stanford benchmark also reveals the real difficulty. Early research found that cross-application data transfer and document handling are major sources of agent failure. Even when an agent understands patient information, it may still make mistakes when uploading files, filling in fields, or updating status. In healthcare, an error in one system can also be copied into multiple downstream systems. That’s exactly why manual approval and post-action state verification can’t be skipped.

How Can Developers Evaluate the Cost and Feasibility of a Medical Agent?

For developers, the first step doesn’t have to be jumping straight into Computer Use. A safer approach is to break the workflow apart first: for purely textual steps like information extraction and document drafting, you can validate quality using a standard model API; then evaluate the UI interaction step separately for its success rate. The example below uses synthetic material to validate the textual step of “extracting the fields required for prior authorization.” It’s a technical demo only and doesn’t constitute any medical advice:

import openai

client = openai.OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.apiyi.com/v1"  # APIYI OpenAI-compatible interface
)

resp = client.chat.completions.create(
    model="gpt-4o-mini",  # refer to the platform's actual list of available models
    messages=[{"role": "user", "content": "从以下合成病历中列出影像检查预授权需要核对的字段清单:……"}]
)
print(resp.choices[0].message.content)
View the full code with Token cost accounting
import openai

client = openai.OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.apiyi.com/v1"
)

# Unit price: USD / million tokens — fill in according to the model you actually use
PRICE = {"input": 0.10, "output": 0.50, "cached_input": 0.01}

def run_step(prompt: str, model: str = "gpt-4o-mini") -> dict:
    resp = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": "你只做字段抽取,不给出任何诊疗建议,信息不明确时输出 NEED_HUMAN。"},
            {"role": "user", "content": prompt},
        ],
    )
    u = resp.usage
    cached = getattr(getattr(u, "prompt_tokens_details", None), "cached_tokens", 0) or 0
    cost = ((u.prompt_tokens - cached) * PRICE["input"]
            + cached * PRICE["cached_input"]
            + u.completion_tokens * PRICE["output"]) / 1_000_000
    return {"text": resp.choices[0].message.content, "cost_usd": round(cost, 6)}

steps = ["合成病历片段 A ……", "合成病历片段 B ……"]
total = 0.0
for s in steps:
    r = run_step(s)
    total += r["cost_usd"]
    print(r["text"])
print("模型 Token 成本合计(不含工具、环境与维护):", round(total, 6))

🚀 Quick validation tip: Before integrating with a real medical system, you can use a single API key through APIYI at apiyi.com to compare how multiple models perform on textual steps like field extraction and document drafting. New users get free testing credits, which makes it a good way to get your evaluation scripts running first and accumulate task-level data.

On the cost side, here are the Standard short-context prices (USD per million tokens) as of November 17, 2026: GPT-6 Astra input 10, output 50, cached input 1; GPT-6 Luna input 0.10, output 0.50, cached input 0.01. Processing speed, context length, and region all affect actual billing. Working backward from Luna’s $0.085 per task and the “98% lower” ratio, Astra comes out to roughly $4.25 per task — but that’s an estimate, and it reflects cost under baseline conditions only, not a production quote.

A complete accounting should tally visual and text input and billable output separately for each round, then add tool calls, runtime environment, and engineering maintenance. If retry tokens are already included in the total, don’t add them again. The nearly 50x price gap between the two tiers also suggests a layered approach: hand complex reasoning to the high-capability model, give structured, high-frequency steps to the low-cost model, and for workflows that have been explored and stabilized, harden them into deterministic code the way CVS did.

💰 Cost optimization tip: When building model tiers, you can start on APIYI apiyi.com by using the same OpenAI-compatible code to switch between models at different price points, measure quality and token consumption for each subtask, and then decide which parts are worth using a high-capability model.

Validation stage Questions to answer Suggested evidence Reference tools
Text stage Is extraction and drafting quality stable? Human annotation comparison, NEED_HUMAN trigger rate APIYI apiyi.com multi-model comparison
UI operations What is the end-to-end completion rate? Screenshots and traces, post-operation state verification OpenAI Computer Use docs and sandbox environment
Process codification Which steps can be converted into deterministic code? Successful trajectory reproduction rate Two-stage architecture like CUA Agent Kit
Production governance Do permissions and approvals cover critical actions? RBAC configuration, audit log spot checks The organization’s own permission and compliance systems

Are Healthcare Agents and RPA Substitutes or a Convergence?

Traditional RPA relies on fixed rules and UI locators, and scripts can break as soon as the interface changes. Computer-use Agents can understand natural language, observe the current interface, and adapt to changes, but re-reasoning each time introduces cost and variability in results. In a study comparing the two technologies (DOI: 10.1080/23311975.2026.2724760), each technology was run 10 times for each of the P2 and P3 workflows; for the P1 workflow, the Agent had only 1 failure record, which was not included in subsequent statistics, so we can’t broadly claim that all tasks were executed 10 times. The study used a 2025 tool configuration, so it can’t directly represent the performance of 2026 models.

In my view, CVS’s “AI exploration, code codification, repeated execution” approach is more likely to become the mainstream engineering pattern: Agents discover and understand unfamiliar processes, deterministic code reliably repeats known processes, APIs continue to handle high-frequency, structured interactions, and human approval guards critical checkpoints. Combining APIs, Computer Use, deterministic code, and human approval is more realistic than betting on any single technology.

What Do Medical Agents Mean for the Entire AI Industry?

Healthcare is just a high-barrier testing ground. Its fragmented systems, compliance requirements, and liability boundaries are all stricter than those of typical enterprises. If execution-capable agents can establish auditable, pausable, permission-aligned ways of operating here, that approach can be carried over to insurance, finance, government, and other industries that also depend on legacy systems.

It also means the dimensions of industry competition are shifting. In the past, the contest was over whether a model could write good content. Going forward, it’ll be more about whether it can get things done in real systems, whether it can stop in time when something goes wrong, and whether every step can be traced and audited. For software vendors, Computer Use lets AI companies take part in workflows without fully controlling the underlying software — that lowers the integration barrier and puts pressure on the old model of “waiting for vendors to open up their APIs.” But high-reliability, high-frequency interactions are still better suited to APIs; Computer Use is more of a complement than a replacement.

A few things are worth watching next: whether OpenAI will publish per-task evaluation data and full experimental configurations; whether the paper version of HealthAdminBench can become a common yardstick for cross-vendor comparison; whether Abridge’s discharge coordination solution can move from development into deployment; and whether production data from more hospitals can prove that end-to-end completion rates are genuinely usable. Until that evidence shows up, demos and scores are just a starting point, not a conclusion.

Medical agents have already proven they can “take action,” but whether they can truly reduce clinical and administrative burden still comes down to their end-to-end completion rates in real workflows, their ability to stop when something goes wrong, and their long-term costs. As the English proverb goes: The proof of the pudding is in the eating — real results have to be proven in practice.


References

  1. OpenAI Official Announcement: Advancing computer-use agents for healthcare teams
    • Link: openai.com/index/advancing-computer-use-for-healthcare-teams
  2. ChatGPT for Healthcare Help Documentation: Product capabilities, BAA, and plugin descriptions
    • Link: help.openai.com/en/articles/20001046-chatgpt-for-healthcare
  3. OpenAI Computer Use Developer Documentation: Computer Tool and Code Execution paths
    • Link: developers.openai.com/api/docs/guides/tools-computer-use
  4. HealthAdminBench Paper: Tasks and metrics in version 2604.09937v1
    • Link: arxiv.org/html/2604.09937v1
  5. CVS Health CUA Agent Kit: Two-phase exploration and consolidation architecture
    • Link: github.com/cvs-health/cua-agent-kit
  6. Fierce Healthcare Report: HLTH26 partners and application scenarios
    • Link: fiercehealthcare.com/ai-and-machine-learning/hlth26-cvs-health-cigna-and-ambience-collaborating-openai-test-computer-use
  7. Cambridge Dictionary: Proverb source verification
    • Link: dictionary.cambridge.org/us/dictionary/english/proof-of-the-pudding-is-in-the-eating

Author: APIYI Technical Team
Technical Exchange: If you’d like to compare how multiple Large Language Models perform on the text portion of agents, feel free to visit APIYI (apiyi.com) to get testing credits. For more integration instructions, check out the docs.apiyi.com documentation center.

Related Articles

Similar Posts