This page is a draft. It isn't in production builds. Set Drafts to Shown in the Brand panel to see it.

Agent anatomy for a healthcare workload ladder

A fictional health system's use cases mapped to the public evals that proxy them, seated on a nested hardware ladder. Demo numbers, derived from real evals, not measured.

Contents

Before anyone buys hardware for agents, two questions need an answer that is not a vendor slide: which of our workloads can this tier actually run, and what does a task cost once it does. This article starts the Harborlight method series: one way to write those answers down, using a fictional regional health system we call Harborlight Health. Every number here is a demo value, derived from named public evals so the grammar is honest, and none of it is a measured result for any organization. The point is the method, not the scores.

Healthcare is the example because its workloads span the whole range: a front-desk intent router is a single-shot task; a prior-authorization workflow is a multi-step agent; a denials research run is long-horizon; a care-team fleet is many agents at once. The same grammar works for any sector.

In this series

Path through the method: this ladder → system swarm → ladder eval figures → Agentic Operation, in plain language → Evaluations.

Harborlight is also the specimen name on /evals/demo. That page is a different artifact: a proprietary-shaped 12-agent intake-to-discharge suite. This series is seating and population charts over public evals. Same fictional health system name; not the same pack.

The three layers of an agent anatomy

  1. Use cases → evals. Each workload is matched to the nearest public evaluation we actually hold on disk, with the gap stated where none exists. Healthcare has several gaps: there is no clinical-note gold, no medical-coding gold and no imaging benchmark on disk, and the healthcare rows on public leaderboards are citations, not test sets we can run.
  2. Evals → hardness. Each eval sits on a four-step hardness scale, from single-shot answers (H1) through tool use (H2) and multi-step agents (H3) to long-horizon and multi-agent work (H4).
  3. Hardness → hardware tier. Tiers are nested: a tier seats every eval of the tiers below it plus the ones it is the first to seat. A rack does not replace the front-desk PC's evals; it carries them too.

Seventeen demo evals carry the Harborlight use cases across five tiers, from a clinic-edge PC to a dense accelerator rack. The hardware spine is a single-vendor accelerator ladder so that cost per task is comparable along it; the method is the same for any vendor's ladder.

Use cases and the evals that stand in for them

Table 1Harborlight use cases, the public evals that stand in for them, and what is missing. Imaging triage and medication safety have no seat at all: nothing we hold proxies them, so they stay off the ladder rather than get an invented one.

Use caseHardnessDerived from (held on disk unless noted)Gap
Portal FAQ / intent routingH1ComplexFuncBench · τ²-bench (short slices)No portal intent gold
Grounded patient FAQ (never diagnostic)H1SimpleQA Verified · TruthfulQA · HealthBench (public gallery cite only)No clinical-advice safety gold
Scheduling / remindersH2τ²-bench · AutomationBench (public split)Booking shape only
Policy / formulary freshnessH1FreshQAFreshness shape only
Patient messaging tool callsH2ComplexFuncBenchNo messaging gold
Ambient note draftH3RAGTruth · MedScribe (public gallery cite only)No clinical-note gold on disk
Coding assistH3SOPBench · MedCode (public gallery cite only)No coding gold on disk
Prior-authorization workflowH3τ²-bench · SOPBench · AutomationBenchNo payer gold
Multi-record chart reviewH3FRAMES · MultiHop-RAGNo longitudinal chart pack
Operations desk ticketsH3EnterpriseOps-GymCommons floor, not hospital-ops gold
Denials researchH4FinAgent · FinSearchCompNo claims gold
Longitudinal record digestH4LongBench v2 · HELMET (catalogued, not fetched)No long-record clinical gold
Integration / access-audit runbooksH4Terminal-Bench 4.0 · SOPBenchNo audit episode pack
Care-coordination across appsH4AppWorld · τ²-benchNo care-team gold
Guideline research briefH4ReportBench · FRAMESNo guideline gold
Care-team fleet densityH4Concurrency over τ²-bench, AutomationBench, EnterpriseOps-GymDensity is a measurement, not a bench
System-wide orchestrationH4EnterpriseOps-Gym + Terminal-Bench as a rack soakNo orchestration bench

The inventory behind this is small and we say so: as of 2 November 2026 the machine index holds 162 candidate evals, 71 of them named in the catalog and 33 on disk, and the number of these we have measured on any organization's hardware is zero.1 The three healthcare names above (MedScribe, MedCode, HealthBench) appear on public leaderboards; we cite them as lineage for a spoke and nothing more.

Detailed eval figures

The seating matrix, the per-tier radar, and the cost-against-success scatter are interactive and heavy, so they live on a companion page: Device-ladder eval figures. That pack shows nested seating (which tier first seats each eval), the same spoke set for every tier so shapes are comparable, and one-eval-at-a-time cost against success on a log scale. Same Harborlight specimen, same demo numbers, same honesty labels.

The population view - roles, composite tasks, and who burns the tokens on the same specimen - is a separate article: Healthcare system swarm. The operating vocabulary behind seats, empty seats, and estimate-vs-score is in Agentic Operation, in plain language. For how an evaluation engagement is sold and run, see Evaluations.

What a real run would change

Nothing on this page is measured. The demo numbers exist so the charts have the right shape and the right honesty labels. For an actual organization the method is: take the use-case table, swap the fictional specimen for the real workload list, run the held evals (and the organization's own task packs beside them) on the candidate tiers inside the organization's governed environment, and replace each demo cell with a measured one, keeping the hollow cells hollow until there is a test set to fill them.

Notes

  1. Counts are from our eval inventory as of 2026-11-02: 162 machine-indexed candidates, 71 named catalog rows, 33 on disk, 0 measured on any organization's hardware. Public leaderboard rows for healthcare are citations only; none is a test set we hold. The figures read the public subset of the ladder pack's score file; the "Show the data" table under each figure is that file. ↩︎

How can we help?

Name the problem you're working on. Shaun typically replies within two working days, by email, with the consultants in the practice who have solved that shape of problem before.

Topic (optional)