Agent anatomy of a healthcare system: population, composite tasks and who burns the tokens

The companion article answered a narrow question: which hardware tier can seat which workload, one eval at a time. This one widens the frame to the thing a health system actually runs: a population of agents, each with a role, a minimum tier, a place in a handful of composite tasks, and a share of the tasks and tokens. The specimen is the same fictional regional system, Harborlight Health, on the same five nested tiers from a clinic-edge PC to a dense accelerator rack. Every number is a demo value, derived from named public evals so the grammar is honest; none of it is a measured result for any organization, and the task and token shares are an illustrative mix, not a trace.

In this series

Path: device ladder → this swarm → ladder eval figures → Agentic Operation, in plain language → Evaluations.

Harborlight is also the specimen name on /evals/demo. That page is a different artifact: a proprietary-shaped 12-agent intake-to-discharge suite. This page is a population and composite-task view over public evals. Same fictional health system name; not the same pack.

Why a population, not a model

A "healthcare AI" procurement conversation usually collapses to one question about one model. A health system's work does not look like that. A portal request is routed, booked and confirmed by three short, cheap roles. An encounter transcript is drafted, criticised, coded and criticised again before a clinician signs it. A denied claim is researched for hours by one agent and checked in minutes by another. The useful unit of analysis is the role: what it does, how hard that is, the smallest tier it can sit on, who checks it, and how much of the system's work and spend it takes.

Four ideas carry the rest of the article:

1. Hardness, not size. Four steps, from single-shot answers (H1) through tool loops (H2) and multi-step agents (H3) to long-horizon and multi-agent work (H4). Each step first seats at a tier and stays seated above it.
2. Roles seat where their evals seat. A role's minimum tier is the first tier at which the public evals that proxy its work seat on the companion ladder. The ladder's seats are demo; so are these.
3. Composite tasks are the orchestration grammar. Six of them cover most of what Harborlight would run; each is a short sequence of roles with a human gate at the end.
4. Tasks and tokens are different distributions. The roles that do most of the tasks are not the roles that burn most of the tokens, and the gap between those two curves is where the hardware decision lives.

The hardness spectrum, seen from the system

Figure 1Four hardness steps against five tiers

demo · derived from real evals · not Measured Harborlight Health · fictional regional health system · specimen · illustrative · not a client

H1 Single-shot answerH2 Tool loopH3 Multi-step agentH4 Long-horizon · multi-agent · fleet

H3 Multi-step agent · harness loops · grounded drafts · constraints · multi-document

A harness plus model that drafts, checks and revises: scribe, coder, critic, prior-auth preparer, chart reviewer.

Composite example: Encounter documentation loop (draft → critique → code → accept → sign). Evals at this step:

• AutomationBench (public split) partial
• RAGTruth on disk
• SOPBench on disk
• FRAMES on disk
• MultiHop-RAG on disk
• EnterpriseOps-Gym on disk
• Mind2Web (offline test) on disk
• ALCE on disk

Show the data

Data for figure 1: Four hardness steps against five tiers. Hardness steps, where each first seats, what it means in the system and the evals at that step. Demo seating.

Step	Agentivity	First seats at	Composite example	Evals
H1 Single-shot answer	one turn · no or light tools	T1	Patient-access episode	SimpleQA Verified, TruthfulQA, FreshQA, FACTS Grounding (public)
H2 Tool loop	a few tool calls · short horizon · light state	T1	Patient-access episode	ComplexFuncBench, τ²-bench, TableBench, AttributionBench
H3 Multi-step agent	harness loops · grounded drafts · constraints · multi-document	T2	Encounter documentation loop	AutomationBench (public split), RAGTruth, SOPBench, FRAMES, MultiHop-RAG, EnterpriseOps-Gym, Mind2Web (offline test), ALCE
H4 Long-horizon · multi-agent · fleet	hours-long episodes · many concurrent agents · orchestration	T3	Care-team coordination fleet	AppWorld, FinAgent (SEC EDGAR), ReportBench, LongBench v2, Terminal-Bench 4.0

Two things are easy to miss. First, most of the roles in a health system are H1-H3 and would seat below a single accelerator; the accelerator and rack tiers exist for the long-horizon and fleet roles, not for the front desk. Second, the step is about the task, not the model: the same open-weight model can be an H1 answerer in one role and the brain inside an H3 harness in another.

The agent population

Seventeen roles make up the Harborlight population. Three are patient-access roles that live on the clinic edge; six are deskside roles for documentation, coding, prior authorization, chart review and the operations desk; five are long-horizon roles that first need a single accelerator; three are the supervisors, watchers and the one orchestrator that need mid-serve density or the rack.

Figure 2Roles by minimum tier, with what motivates each seat

demo · derived from real evals · not Measured Harborlight Health · fictional regional health system · specimen · illustrative · not a client

Role · pick one for its evals	Hardness	T1clinic edge	T2dept deskside	T3single accel.	T4mid-serve	T5rack
First seat T1 · Clinic edge · Ryzen AI / Radeon
R01 Front-desk intent router	H1	S	·	·	·	·
R02 Scheduling assistant	H2	S	·	·	·	·
R03 Patient-message drafter	H2	S	·	·	·	·
First seat T2 · Department deskside · Radeon PRO
R04 Scribe drafter	H3	—	S	·	·	·
R05 Coding suggester	H3	—	S	·	·	·
R06 Faithfulness critic / acceptor	H3	—	S	·	·	·
R07 Prior-auth preparer	H3	—	S	·	·	·
R08 Chart reviewer	H3	—	S	·	·	·
R09 Ops-desk resolver	H3	—	S	·	·	·
First seat T3 · Single Instinct · MI300X/MI325X
R10 Denials researcher / appeal writer	H4	—	—	S	·	·
R11 Record digester	H4	—	—	S	·	·
R12 Guideline librarian	H4	—	—	S	·	·
R13 Interface runbook engineer	H4	—	—	S	·	·
R14 Care-coordination broker	H4	—	—	S	·	·
First seat T4 · Multi-GPU mid-serve
R15 Care-team fleet supervisor	H4	—	—	—	S	·
R16 Audit / telemetry watcher	H4	—	—	—	S	·
First seat T5 · Dense Instinct rack / UBB
R17 System-wide orchestrator	H4	—	—	—	—	S

S minimum tier (first seat) · · also runs here (nested) · — not seated. 17 roles; a role's share of tasks and tokens is in its detail and in the distribution figure.

R06 Faithfulness critic / acceptor · first seats at T2 (dept deskside) · H3 · mid burn · 10% of tasks · 7% of tokens (demo)

Checks every draft against its sources before a human sees it. Appears in four of six composites, which is why it has many tasks at moderate burn.

Evals that motivate the seat (companion ladder first seats: HDL-C06):

• FACTS Grounding (public) on disk
• RAGTruth on disk
• AttributionBench on disk

In composites: Encounter documentation loop · Prior-authorization packet · Denial appeal research · Interface and access-audit runbook. Human gate: its verdict is advice to the human gate, not the gate.

Show the data

Data for figure 2: Roles by minimum tier, with what motivates each seat. Roles with minimum tier, hardness, motivating evals, composite tasks, and demo share of tasks and tokens.

Role	Min tier	Hardness	Evals	Composites	Tasks %	Tokens %	Human gate
R01 Front-desk intent router	T1	H1	ComplexFuncBench, τ²-bench	X01	14	3	exceptions go to the front desk
R02 Scheduling assistant	T1	H2	τ²-bench, AutomationBench (public split), Natural Plan (calendar scheduling)	X01	12	4	scheduler confirms overrides
R03 Patient-message drafter	T1	H2	ComplexFuncBench, FACTS Grounding (public), EmailBench	X01, X06	10	4	staff send; never clinical advice
R04 Scribe drafter	T2	H3	RAGTruth, FACTS Grounding (public), MedScribe	X02	9	16	clinician signs every note
R05 Coding suggester	T2	H3	SOPBench, MedCode	X02	7	6	a coder signs every code
R06 Faithfulness critic / acceptor	T2	H3	FACTS Grounding (public), RAGTruth, AttributionBench	X02, X03, X04, X05	10	7	its verdict is advice to the human gate, not the gate
R07 Prior-auth preparer	T2	H3	τ²-bench, SOPBench, Mind2Web (offline test)	X03	5	8	staff submit the packet
R08 Chart reviewer	T2	H3	FRAMES, MultiHop-RAG, TableBench	X03	4	6	clinician reads the summary
R09 Ops-desk resolver	T2	H3	EnterpriseOps-Gym, TableBench	X05	6	4	desk staff close tickets
R10 Denials researcher / appeal writer	T3	H4	FinAgent (SEC EDGAR), ReportBench, ALCE	X04	3	10	revenue-cycle staff review and send
R11 Record digester	T3	H4	LongBench v2, FRAMES	X04, X03	2	8	clinician reads the digest
R12 Guideline librarian	T3	H4	ReportBench, ALCE, FRAMES	X04	2	5	committee reads the brief
R13 Interface runbook engineer	T3	H4	Terminal-Bench 4.0, SOPBench, ITBench-AA	X05	2	7	security staff approve each remediation
R14 Care-coordination broker	T3	H4	AppWorld, τ²-bench	X06	5	6	care manager owns the plan
R15 Care-team fleet supervisor	T4	H4	τ²-bench, EnterpriseOps-Gym, TheAgentCompany (inspect_evals pointer)	X06	4	2	care-management lead
R16 Audit / telemetry watcher	T4	H4	Terminal-Bench 4.0, ITBench-AA	X05, X06	4	3	security and operations staff
R17 System-wide orchestrator	T5	H4	EnterpriseOps-Gym, Terminal-Bench 4.0, TheAgentCompany (inspect_evals pointer)	X06	1	1	platform team
Total					100	100

The critic deserves the attention. It appears in four of the six composite tasks, so it carries a tenth of all tasks at moderate burn, and it is the role that turns "the model drafted something" into "a human can safely read this". Its proxies are the grounding and attribution evals (FACTS Grounding, RAGTruth, AttributionBench), all of which we hold, which makes it one of the better-evidenced roles in the system despite having no healthcare gold behind it.

Six composite tasks as swimlanes

Figure 3One composite task at a time, steps in the lane of the tier they need

demo · derived from real evals · not Measured Harborlight Health · fictional regional health system · specimen · illustrative · not a client

X01 Patient-access episodeX02 Encounter documentation loopX03 Prior-authorization packetX04 Denial appeal researchX05 Interface and access-audit runbookX06 Care-team coordination fleet

X02 Encounter documentation loop · H3 · draft → critique → code → accept → sign

An encounter transcript becomes a signed note and a set of suggested codes.

1. Scribe drafter T2 — draft the note from the transcript
2. Faithfulness critic / acceptor T2 — check every sentence against the transcript; send back what is unsupported
3. Coding suggester T2 — suggest codes with rule citations from the accepted draft
4. Faithfulness critic / acceptor T2 — check each code's citation
5. Human gate — clinician signs the note; a coder signs the codes

Gap: No clinical-note or coding gold on disk; MedScribe and MedCode are leaderboard names only.

Show the data

Data for figure 3: One composite task at a time, steps in the lane of the tier they need. Every composite task, step by step: the role, the lowest tier it needs, and the action. Demo seating.

Composite	Step	Role	Tier	Action
X01 Patient-access episode (H2)	1	Front-desk intent router	T1	classify the request and pick the tool or queue
	2	Scheduling assistant	T1	book, move or remind under scheduling rules
	3	Patient-message drafter	T1	draft the confirmation from grounded sources
	4	Human gate	—	front desk handles exceptions and anything clinical
X02 Encounter documentation loop (H3)	1	Scribe drafter	T2	draft the note from the transcript
	2	Faithfulness critic / acceptor	T2	check every sentence against the transcript; send back what is unsupported
	3	Coding suggester	T2	suggest codes with rule citations from the accepted draft
	4	Faithfulness critic / acceptor	T2	check each code's citation
	5	Human gate	—	clinician signs the note; a coder signs the codes
X03 Prior-authorization packet (H3)	1	Prior-auth preparer	T2	run the eligibility checklist under payer policy
	2	Chart reviewer	T2	pull the supporting encounters, results and tables
	3	Prior-auth preparer	T2	work the payer portal and assemble the packet
	4	Faithfulness critic / acceptor	T2	check the packet against the checklist and the evidence
	5	Human gate	—	authorization staff submit
X04 Denial appeal research (H4)	1	Denials researcher / appeal writer	T3	research the contract, policy and remittance history
	2	Record digester	T3	digest the relevant record history
	3	Guideline librarian	T3	find the guideline passages that support the case
	4	Denials researcher / appeal writer	T3	write the appeal with citations
	5	Faithfulness critic / acceptor	T2	verify every citation supports its claim
	6	Human gate	—	revenue-cycle staff review and send
X05 Interface and access-audit runbook (H4)	1	Interface runbook engineer	T3	execute the runbook step by step in the governed environment
	2	Faithfulness critic / acceptor	T2	verify pre- and post-conditions
	3	Ops-desk resolver	T2	open or close the operations tickets the run produces
	4	Audit / telemetry watcher	T4	watch logs and queues continuously
	5	Human gate	—	security staff approve each remediation
X06 Care-team coordination fleet (H4)	1	Care-team fleet supervisor	T4	start one broker per episode and watch the concurrency cliff
	2	Care-coordination broker	T3	coordinate one episode across apps
	3	Patient-message drafter	T1	draft the patient and team messages
	4	Audit / telemetry watcher	T4	watch agent health and queues
	5	System-wide orchestrator	T5	place and fail over across nodes at system scale
	6	Human gate	—	care manager owns each plan

Reading the lanes tells you something a tier table cannot: how much of a task's path is cheap. The documentation loop never leaves the deskside lane. The prior-authorization packet is deskside work too, with a web-GUI step that the inventory covers better than most people expect (Mind2Web). Only the appeal research and the fleet reach the accelerator lanes, and only the orchestrator reaches the rack. A buyer who sizes the whole population at the rack tier is paying for the lane the fewest steps use.

Who does the tasks and who burns the tokens

Figure 4Share of tasks and share of tokens, per role

demo · derived from real evals · not Measured Harborlight Health · fictional regional health system · specimen · illustrative · not a client

by first seatby tasks %by tokens %

By hardness step · tasks % → tokens %

• H1 Single-shot answer · 1 roles · 14% → 3%
• H2 Tool loop · 2 roles · 22% → 8%
• H3 Multi-step agent · 6 roles · 41% → 47%
• H4 Long-horizon · multi-agent · fleet · 8 roles · 23% → 42%

Illustrative mix for the whole Harborlight population. Patient-access roles take about a third of the tasks on about a tenth of the tokens; the documentation loop takes a quarter of the tasks for a bit under a third of the tokens; the five long-horizon roles take a seventh of the tasks for over a third of the tokens; supervisors and watchers are thin.

Show the data

Data for figure 4: Share of tasks and share of tokens, per role. Roles with minimum tier, hardness, motivating evals, composite tasks, and demo share of tasks and tokens.

Role	Min tier	Hardness	Evals	Composites	Tasks %	Tokens %	Human gate
R01 Front-desk intent router	T1	H1	ComplexFuncBench, τ²-bench	X01	14	3	exceptions go to the front desk
R02 Scheduling assistant	T1	H2	τ²-bench, AutomationBench (public split), Natural Plan (calendar scheduling)	X01	12	4	scheduler confirms overrides
R03 Patient-message drafter	T1	H2	ComplexFuncBench, FACTS Grounding (public), EmailBench	X01, X06	10	4	staff send; never clinical advice
R04 Scribe drafter	T2	H3	RAGTruth, FACTS Grounding (public), MedScribe	X02	9	16	clinician signs every note
R05 Coding suggester	T2	H3	SOPBench, MedCode	X02	7	6	a coder signs every code
R06 Faithfulness critic / acceptor	T2	H3	FACTS Grounding (public), RAGTruth, AttributionBench	X02, X03, X04, X05	10	7	its verdict is advice to the human gate, not the gate
R07 Prior-auth preparer	T2	H3	τ²-bench, SOPBench, Mind2Web (offline test)	X03	5	8	staff submit the packet
R08 Chart reviewer	T2	H3	FRAMES, MultiHop-RAG, TableBench	X03	4	6	clinician reads the summary
R09 Ops-desk resolver	T2	H3	EnterpriseOps-Gym, TableBench	X05	6	4	desk staff close tickets
R10 Denials researcher / appeal writer	T3	H4	FinAgent (SEC EDGAR), ReportBench, ALCE	X04	3	10	revenue-cycle staff review and send
R11 Record digester	T3	H4	LongBench v2, FRAMES	X04, X03	2	8	clinician reads the digest
R12 Guideline librarian	T3	H4	ReportBench, ALCE, FRAMES	X04	2	5	committee reads the brief
R13 Interface runbook engineer	T3	H4	Terminal-Bench 4.0, SOPBench, ITBench-AA	X05	2	7	security staff approve each remediation
R14 Care-coordination broker	T3	H4	AppWorld, τ²-bench	X06	5	6	care manager owns the plan
R15 Care-team fleet supervisor	T4	H4	τ²-bench, EnterpriseOps-Gym, TheAgentCompany (inspect_evals pointer)	X06	4	2	care-management lead
R16 Audit / telemetry watcher	T4	H4	Terminal-Bench 4.0, ITBench-AA	X05, X06	4	3	security and operations staff
R17 System-wide orchestrator	T5	H4	EnterpriseOps-Gym, Terminal-Bench 4.0, TheAgentCompany (inspect_evals pointer)	X06	1	1	platform team
Total					100	100

The shape is the point, not the digits. The three patient-access roles take about a third of the tasks on about a tenth of the tokens: numerous, short, and cheap enough to stay on premises at the clinic edge. The documentation loop takes a quarter of the tasks for a bit under a third of the tokens, almost all of it in the scribe drafter, whose inputs and outputs are long and whose drafts get revised. The five long-horizon roles take a seventh of the tasks for over a third of the tokens: few episodes, each long. Supervisors, watchers and the orchestrator are thin on both axes; their cost is machine time and consequence, not tokens. Once an organization replaces this mix with its own trace, the hardware question becomes arithmetic: the burn cluster wants owned accelerator time, the access roles want cheap edge seats, and nothing in between needs a rack.

What the inventory honestly holds, per domain

Figure 5Use-case density against the evals we can point at

demo · derived from real evals · not Measured Harborlight Health · fictional regional health system · specimen · illustrative · not a client

Domain · pick one for its evals	use cases	evals we can point at, by status	density
Patient accessPortal FAQ / intent routing · Scheduling / reminders · Patient messaging · Policy / hours freshness	4	6 on disk · 1 partial · 1 cite only · 2 index only	dense shape · hollow gold
Clinical documentation and codingAmbient note draft · Coding assist · Multi-record chart review	3	7 on disk · 2 cite only	shape only · hollow gold
Access and revenuePrior-authorization packet · Denial appeal research	2	6 on disk	thin · hollow gold
Records, operations and integrationLongitudinal record digest · Operations desk · Interface / access-audit runbooks	3	4 on disk · 1 cite only	thin · hollow gold
Care coordination and fleetCare-coordination across apps · Care-team fleet density · System-wide orchestration	3	3 on disk · 1 index only	thin · hollow gold
Imaging and medication safetyImaging triage / report draft · Medication reconciliation	2	—	no seat

Counts are of the evals this system anatomy points at, by status, not catalog totals. Inventory as of 2026-11-02: 162 candidates · 71 named · 33 on disk · 0 measured on any organisation's hardware.

Show the data

Data for figure 5: Use-case density against the evals we can point at. Each eval this system anatomy points at, its inventory status, source, hardness, why it fits, and its gap.

Eval	Status	Source	Hardness	Why it fits	Gap	In the ladder article
ComplexFuncBench	ON_DISK	catalog #3 · 1000	H2	Multi-step function calling with argument fidelity and stop conditions: the shape of routing, messaging and lookup tools at the front desk.	Travel-domain tools, not clinical or scheduling systems.	yes
τ²-bench	ON_DISK	catalog #2 · airline 50 / retail 114	H3	User-facing dialog under written policy with a database behind it: scheduling, prior-auth eligibility steps, care-coordination handoffs.	Airline and retail policies, not payer or scheduling policy.	yes
AutomationBench (public split)	PARTIAL	catalog #1 · public split on disk; AA set private	H3	Short automation paths across consumer apps; a floor for scheduling and reminder flows.	Public split only; no healthcare apps.	yes
SimpleQA Verified	ON_DISK	catalog #13 · 1000	H1	Short grounded factual answers, the floor for a patient-facing FAQ that must never improvise.	General knowledge, not patient-facing content.	yes
TruthfulQA	ON_DISK	catalog #18 · 817	H1	Resisting plausible falsehoods; stands in for refusing clinical-advice prompts.	Not a clinical-safety set.	yes
FreshQA	ON_DISK	catalog #8 · 500 test	H1	Answers that change over time: hours, formulary, policy lookups.	Freshness shape only; no formulary corpus.	yes
RAGTruth	ON_DISK	catalog #5 · 17790 responses	H3	Hallucination rate in grounded drafts: the headline metric for an ambient note and the critic that checks it.	News / QA sources, not encounter transcripts.	yes
SOPBench	ON_DISK	catalog #44 · 903 tasks · 167 tools	H3	Following a written procedure with tools and refusing out-of-policy steps: coding rules, prior-auth checklists, runbook steps.	Commercial-service procedures, not clinical or payer procedures.	yes
FRAMES	ON_DISK	catalog #6 · 824	H3	Reasoning across several documents, the shape of a multi-encounter chart review.	Wikipedia hops, not a longitudinal record.	yes
MultiHop-RAG	ON_DISK	catalog #21 · 2556 queries	H3	Retrieval across many sources before answering; pairs with FRAMES for chart review.	News corpus.	yes
EnterpriseOps-Gym	ON_DISK	catalog #11 · cite oracle 649	H3	Ticket resolution with tools across an enterprise: bed flow, staffing and IT desk tickets at the commons level.	Enterprise commons floor, not hospital-operations gold.	yes
AppWorld	ON_DISK	catalog #27 · 732 task dirs	H4	One agent working across several apps with persistent state: a care-coordination broker's shape.	Consumer apps, not clinical systems.	yes
FinAgent (SEC EDGAR)	ON_DISK	catalog #70 · 133 graded	H4	Research over long regulatory filings with graded evidence: the shape of denials and appeal research over contracts, policies and remittances.	SEC filings, not payer documents; no claims gold.	yes
ReportBench	ON_DISK	catalog #29 · 100	H4	Research-to-report quality with citations: guideline briefs and appeal letters.	General research topics.	yes
LongBench v2	ON_DISK	catalog #24 · 503 · 8k–2M words	H4	Reasoning over very long inputs: a longitudinal record digest.	Not clinical text.	yes
Terminal-Bench 4.0	ON_DISK	catalog #10 · 66 tasks	H4	Long shell tasks with verification: interface checks, access-log audits, rehearsed remediations.	No access-audit episode pack.	yes
MedScribe	cite-gallery	vals.ai Healthcare row · SOAP / clinical documentation · Accuracy vs Cost/Test	H3	The closest public name to an ambient note; lineage label for the scribe role only.	Not held on disk; not a test set we can run.	yes
MedCode	cite-gallery	vals.ai Healthcare row · ICD-10-CM billing	H3	The closest public name to coding assist; lineage label for the coding role only.	Not held on disk.	yes
HealthBench	cite-gallery	AA promoted list · gallery charts only	H1	Lineage label for the patient-facing FAQ refusal behaviour.	No catalog row; gallery name only.	yes
FACTS Grounding (public)	ON_DISK	catalog #4 · 860	H1	Is the answer supported by the supplied document? The critic / acceptor role's primary proxy, and a floor for grounded patient replies.	General documents, not clinical.	added
TableBench	ON_DISK	catalog #41 · 886	H2	Reasoning over tables: lab and vitals flowsheets in chart review, census and staffing tables at the ops desk.	Open-web tables, not flowsheets.	added
Mind2Web (offline test)	ON_DISK	catalog #47 · 1341 test tasks	H3	Completing tasks on real web interfaces: the payer-portal half of a prior-authorization packet.	Offline test only; online split gated; no payer portals.	added
ALCE	ON_DISK	catalog #32 · ASQA / ELI5 / QAMPARI	H3	Every claim carries a citation that supports it: appeal letters and guideline briefs.	Open-domain QA, not clinical literature.	added
AttributionBench	ON_DISK	catalog #42 · merged test 2672	H2	Does this reference support this claim? A cheap check the critic runs on every cited draft.	General claims.	added
ITBench-AA	cite-gallery	AA promoted list · IT / SRE / security ops tasks · gallery charts only	H4	Lineage label for the interface-runbook and audit-watcher roles.	Gallery name only; not held.	added
EmailBench	machine-index	candidates.jsonl batch 2 / 15 · PARTIAL · 206 scenarios claimed · no licence file · not fetched	H2	Patient-message triage is an inbox problem; the nearest named shape, but only an index row today.	Not in the named catalog; not fetched; no healthcare inbox.	added
Natural Plan (calendar scheduling)	machine-index	candidates.jsonl batch 4 · VERIFIED · not in the named catalog · not fetched	H2	Calendar scheduling under constraints; the planning half of appointment and staff scheduling.	Index row only; not fetched; no clinic constraints.	added
TheAgentCompany (inspect_evals pointer)	machine-index	candidates.jsonl batch 15 · inspect_evals thin-axis catalog row · not fetched	H4	A multi-agent company simulation is the nearest public shape to fleet supervision and orchestration.	Pointer row only; not fetched; not a hospital.	added

This figure is the honesty ledger for the whole article. The patient-access and documentation domains have the densest shape coverage (grounding, tool-use, procedure-following and multi-hop evals we actually hold) and no domain gold at all: no clinical-note set, no coding set, no clinical-advice safety set on disk. The healthcare names that do exist publicly (MedScribe, MedCode, HealthBench) are leaderboard citations, not test sets we can run, and they appear here as lineage labels on three roles and nothing more. The inventory behind this is small and we say so: as of 2 November 2026 the machine index holds 162 candidate evals, 71 of them named in the catalog and 33 on disk, and the number of these measured on any organization's hardware is zero.¹

What changed in the eval set

Re-reading the inventory for an agent system rather than a seating ladder added nine names and dropped two. Added, all held on disk: FACTS Grounding and AttributionBench for the critic; ALCE for cited appeals and briefs; TableBench for flowsheets and census tables; Mind2Web for the payer-portal step. Added as citations only: ITBench for the runbook and watcher roles, and three machine-index rows that are the nearest public shapes to inbox triage, calendar scheduling and a many-agent company simulation, none of them fetched. Dropped: a markets-search set whose fit to healthcare was thin once a document-research set already covered the shape, and a long-context set that is catalogued but not fetched. The use-case to eval map with each decision is in the pack behind this article.

What a real run would change

Nothing on this page is measured. The method for an actual organization is the same as the companion article's, one layer up: replace the fictional population with the organization's own role list, replace the illustrative task and token mix with a week of traced work, run the held evals (and the organization's own task packs beside them) on the candidate tiers inside the organization's governed environment, and let the measured seats move roles up or down the ladder. Keep the hollow domains hollow until there is a test set to fill them.

Related

• Healthcare device ladder - use cases, stand-in evals, and nested hardware seating.
• Device-ladder eval figures - seating matrix, radar, and cost-against-success.
• Agentic Operation, in plain language - seats, empty seats, and estimate vs score.
• Evaluations · Evals demo figures - engagement shape and the proprietary 12-agent Harborlight suite (not this population pack).