Skip to content

Conformance scenario authoring ​

How to add a runtime-conformance scenario and edit your own finding records. Read packages/e2e/src/conformance/catalogue/README.md first — it states the record anatomy and the 3-step bucket lifecycle; this guide does not repeat or contradict it, only expands on it. Background: the design spec docs/superpowers/specs/2026-09-23-runtime-conformance-suite-design.md (§4.1 rules, §7 catalogue/ledger, §9 harness testing/flake policy). The worked example throughout is packages/e2e/src/conformance/scenarios/transcript.ts.

1. Who edits what ​

The conformance owner, as of 2026-09-25, is the Claude session named conformance-prep-and-do-slice — find it with ListAgents and reach it with SendMessage (cross-session). New findings, harness changes, and a fragment proposed for another finding's record (§4) all go there. That name is a snapshot and will go stale; the durable fallback pointer is the reliability orchestrator session (name starts with reliability-improvements-orchestrator, also as of 2026-09-25) — if the named owner session is gone, ask the orchestrator who owns it now.

A bucket worker (a session fixing one or a few reliability findings) may:

  • Add new files under packages/e2e/src/conformance/scenarios/**, wired into scenarios/index.ts's ALL_SCENARIOS and scenario-manifest.ts's SCENARIO_MANIFEST (kept in lockstep — scenario-manifest.test.ts asserts they agree).
  • Add assertions to an existing scenario file, but only for its own finding ids, and only with the conformance owner's review of the MR (controller ruling; precedent: bucket B8 extending transcript.ts with the RM-29 scenarios). This is not standing permission to restructure someone else's scenario — say what you're adding and why in the MR.
  • Edit only its own finding records under packages/e2e/src/conformance/catalogue/<area>/<ID>.ts.

Everything else is harness internals and belongs to the owner — send a message describing the need, not a patch: types.ts, cell.ts, suite.ts, drivers/**, observation/**, assertions.ts, judge.ts, drift.ts, plan-rule.ts, verify*.ts, reporter.ts, results*.ts, ci-guard.ts, catalogue/{define,index,merge,check}.ts, gaps.ts, findings.ts, ledger.ts, docs.ts, soak.ts, timeout.ts, canaries/**, __tests__/**, src/__tests__/conformance-*.integ.test.ts, scripts/**, harness/**, core/src/llm/scripted-model.ts, the vitest.conformance*.config.ts files, eslint.conformance.config.js, and ci/test-matrix.yml's conformance jobs. If a scenario genuinely needs a new driver primitive or a harness change, that is a request to the owner, not something to build yourself.

2. Anatomy of a scenario ​

Every scenario is a Scenario object (types.ts). Walking transcript.ts's pairedTranscriptScenario:

ts
{
  id: 'transcript.finish-cosiblings',
  title: 'A call co-emitted with __finish__ gets a tool_result (RM-29 shape b)',
  assertions: [
    'transcript.finish-cosiblings.turn-settles',
    'transcript.finish-cosiblings.turn1-completes',
    'transcript.finish-cosiblings.persisted-paired',
    'transcript.finish-cosiblings.turn2-completes',
    'transcript.finish-cosiblings.llm-input-has-turn1-calls',
    'transcript.finish-cosiblings.llm-input-paired',
  ],
  tiers: ['executor'],
  needs: ['start', 'observe'],
  async run({ driver, model, check, settle, recordTiming, cellId }) { /* ... */ },
}
  • id — dotted, scenario-family-prefixed (transcript.*, stop.*).

  • assertions — every named assertion id the scenario may record, including its settle() ids (transcript.ts's SUFFIXES constant, mapped to ${id}.${suffix}). AssertionRecorder throws on an undeclared id, so this list must be exact — both scenario families assert this at construction time (if (JSON.stringify(opts.assertions) !== JSON.stringify(expected)) throw).

  • tiers — observation tiers the scenario runs at: 'executor' (the Node backends), 'host' and 'client' (cf-do and js-chat-host), and 'store' (a session store driven directly, no agent — see "The store tier" and "Lanes" below).

  • needs — DriverCapability[] the scenario calls on driver (start, stop, observe, ...). A driver missing one can't express the scenario: the cell records a declared gap, not a pass or a failure (DECLARED_GAPS in gaps.ts — owner-managed). Two capabilities came with the harness-API additions at the executor tier: 'branch' (driver.branch(agent, message, newSessionId, { fromSessionId, checkpointId? | messageIndex? }, opts?) — starts a run in a NEW session cloned from a source session's checkpoint/message index; pair with requires: 'branching' + a rejection.attempt since it's also a PRODUCT capability, not just a driver one — smoke.branch-completes is the worked example) and 'submit-tool-result' (see the StartOptions.clientTools bullet below).

  • start/branch/resume all accept an optional StartOptions.clientTools: a Record<toolName, { output, at, delayMs }> the driver's built-in client-tool responder uses to answer every pending client-executed call of a listed tool — a scenario never calls submitToolResult by hand. It answers a mixed suspended_step_partial (client calls plus a sub-agent wait in one step) exactly like suspended_client_tool; a suspended_awaiting_children result is reported as-is. The responder follows the run across as many suspend/answer rounds as it takes (each call's delayMs counted from when its suspension was observed) and continues the run the way this backend's real callers do (BackendEnv.clientToolContinuation: 'resume' re-attaches via executor.resume(..., { mode: 'continue' }), 'auto' via executor.getHandle(...) — never both). A pending call whose tool is NOT listed is left alone, so the run's real suspended outcome is what outcome() reports — StartOptions.clientTools requires the 'submit-tool-result' driver capability (requireStartOptions throws a HarnessError otherwise); smoke.client-tool-completes is the worked example.

    • The responder's waits count against the caller's settle() bound on outcome(): every delayMs, the wake wait on a clientToolSubmitWakesRun backend (up to CLIENT_TOOL_WAKE_BOUND_MS, 10s, per round), and the woken-run wait (up to CLIENT_TOOL_WOKEN_RUN_BOUND_MS, 30s). Size the settle as the honest SUM of the waits the path can take in sequence, strictly above every chain-internal bound — smoke.ts' CLIENT_TOOL_SETTLE_BOUND_MS is the worked example — so a chain-internal timeout surfaces as its diagnostic HarnessError rather than a bare settle timeout; size the cell's timeoutMs from it too (scenario-budgets.test.ts).
    • Woken-run ownership (clientToolSubmitWakesRun backends, CP-64): once a submit was accepted, the run it woke belongs to the chain until the persisted session shows that run finished: not 'active' and, while still 'paused', the answered entries have cleared (or the run was already seen 'active' — one that went back to 'paused' with the entry still pending has ended, and the chain's real outcome reports it). When the chain stops following the woken run without a handle covering it (the continuation resume() threw, stop(), dispose), outcome() still waits for that (up to CLIENT_TOOL_WOKEN_RUN_BOUND_MS) — stop() and dispose interrupt/abort it through the last handle the chain obtained. RunHandle.stop()'s doc comment says exactly which run a chain's stop() targets between rounds.
    • A submit answered with anything but accepted ends the chain as terminal: 'threw' with source: 'submitToolResult:<status>' (the product's answer, judged like any outcome).
    • The wake / woken-run wait timeouts, a missing executor.submitToolResult, an 'auto' getHandle returning null, a client-call suspension with no suspended.toolCallIds, and any HarnessError the executor or an adapter throws (e.g. a Cloudflare adapter refusing options.branch) are HarnessErrors: outcome() rejects and the cell crashes — never a judged threw. The crash happens even if the scenario never awaited that outcome(): every run records its harness rejection (RunLog.harnessErrorsSince) and runCell crashes the cell on it.
  • requires / rejection — for a scenario whose positive path needs a PRODUCT capability (e.g. workspaces). On a backend lacking it, the runner substitutes rejection.attempt, which must observe a typed framework_not_supported thrown synchronously or as outcome.thrown — a run that starts and fails late does not count.

  • run(ctx) — the scenario body. ctx.cellId is unique per cell; every session id a scenario mints must be prefixed with it (`${cellId}-s`) so parallel cells never collide. Always await driver.prepare(agent) before driver.start(...). Script the model with model.scriptFor(sessionId, ...) — never a global/shared queue; the scripted model is per-session so parallel cells can't cross-talk. Scripts no run consumed are reported per cell as leftoverScripts (results + the verify summary, beside left-running) — informational only, never a verdict change (a rejection path legitimately scripts a turn the runtime never reaches).

    • model.scriptForAgentType(agentType, ...) scripts a queue shared by every session of agentType (e.g. a spawned sub-agent's type) rather than one session. Because that queue is a single FIFO shared by ALL sessions of that type, only use it when a single session of the type will consume the responses in a cell, or when the responses are interchangeable (order-independent) — otherwise which session gets which response is non-deterministic. Once a specific child session id is known (e.g. read back from Observation.subSessions after the parent spawned it), switch to model.scriptFor(childSessionId, ...) for a deterministic assignment. ctx.model.scriptForAgentType carries the same cell-id prefix guard as session ids: an agentType not prefixed with `${cellId}-` (the - boundary included — cell c may not script cell c2's c2-child) throws HarnessError (cell.ts's guardScenarioModel) — a sub-agent scripted this way must be defined with a per-cell name (e.g. defineAgent({ name: `${cellId}-child` })) and, like the parent, go through await driver.prepare(childAgent) before it can run (Temporal's workflow registry resolves agents by name, so an unprepared/unnamed child agent won't be found there either).

Scenario agents (executor tier) ​

Every executor-tier scenario agent is a bundled factory. The scenario does not call defineAgent itself: it builds its agent from a factory in conformance/agents/<module>.ts (one module per scenario module), registered in AGENTS (agents/registry.ts) by BASE name. The same factories are bundled into the cf-do worker, so one agent definition serves every lane.

  • Static vs per-cell entries. A static entry's agent is named exactly its base (conf-stop). A per-cell entry's agent is ${cellId}-<base> (what scriptForAgentType's guard in cell.ts needs for a sub-agent); its factory takes the cell id and builds a parent's child with the same prefix. An entry is BundledAgentEntry { perCell, build({ cellId?, ports? }) }; a cell id on a static entry, or none on a per-cell one, is a BundledAgentError. Build through buildBundledAgent(base, { cellId }) or the module's exported factory.

  • The transformation rule (inline agent to factory): move the defineAgent call and its tools into agents/<module>.ts as an exported fooAgent(...) function; wrap the result in registryBuilt(agent, { base, cellId? }); give each tool a module-specific export so two same-named tools in different agents stay distinct (fingerprints cover tool schemas); replace any Node-side state a tool body reached (a module-level latch map, a counter) with a PORT call. Abort races stay in the tool body (whether a runtime aborts a tool is product behaviour); the port only awaits release. Keep the scenario's assertion ids, needs and outcomes unchanged.

  • Ports (agents/ports.ts, AgentPorts), handed to the factory by the host:

    • toolGate({ sessionId, gateId }) blocks while the harness holds that gate and resolves at once when it is not held (an unheld gate still counts as entered);
    • witness({ sessionId, toolCallId, tool }) records that the tool's execute body RAN, independent of any state commit.

    A tool built without the port it needs THROWS, so a missing port can never pass an ungated latch or an unwitnessed execution by accident. A port is a function, so wiring one never changes the fingerprint.

  • In-process ports on the executor lanes. The executor driver owns an ExecutorPortControls (in-process ports from harness/in-process-ports.ts plus a HoldLedger) and exposes them as driver.toolGates (hold-tool-gate, driverToolGates(driver)) and driver.toolWitnesses (observe-tool-witness, driverToolWitnesses(driver) with executed(sessionId) and waitUntil(sessionId, toolCallId, boundMs)). Hold the gate BEFORE start, waitUntilEntered to prove the window, and release in a finally. A gate still held at cell end is released and named in a HarnessError by endCell; dispose releases silently (it only meets a held gate after an outer-timeout crash).

  • prepare() rebuilds and verifies. driver.prepare(agent) requires the agent to be registry-built (agents/provenance.ts: the agent object and each tool object came out of a factory, untouched); anything else, an inline defineAgent included, is refused with a HarnessError naming the agent. It then REBUILDS the agent from the registry origin (base, cell id) with the driver's in-process ports, and demands the rebuilt agent's fingerprint equal the scenario's (so a factory must not branch on its ports), then wires the hooks recorder and installs the REBUILT agent. Pass the agent prepare() returned to start/resume/retry.

  • registryMutated is a canary-only seam. A harness canary that must run a deliberately faulty agent builds it with registryMutated(source, mutate) (agents/provenance.ts), which records the mutation in the origin so the rebuild re-applies it. registryBuilt and registryMutated are lint-confined (lint:conformance): scenarios may not import them; only the agent modules (registryBuilt) and canaries/tests (registryMutated) may. Scenario agents are never mutated.

Adding a new executor scenario agent:

  1. Write the factory in agents/<module>.ts (create the module if the scenario module has none), with an exported *_AGENT_NAME constant, registryBuilt around the result, and ports taken as an argument where a tool needs a gate or witness.
  2. Register it in AGENTS (agents/registry.ts) under its base name, using statik(...) or perCell(...).
  3. Bump the count in __tests__/agent-provenance.test.ts (expect(Object.keys(AGENTS)).toHaveLength(N); 54 at MR-1a). That test and the fingerprint test cover every registered factory.
  4. In the scenario, await driver.prepare(fooAgent(...)) and use the returned agent; declare hold-tool-gate / observe-tool-witness in needs when the scenario uses them.
  5. Run test:conformance-unit, test:conformance-harness and lint:conformance.

What exists today (executor tier) ​

Driver capabilities: start · resume · retry · stop · seed · observe · branch · submit-tool-result (0a + the harness-API additions), delete-session (every executor lane; below), plus three added by Task 16A (see "Completion support", "Seeding a companion" and "Run-start commit holds" below): seed-companion (the JS, Temporal and CF Workflows executor lanes; not dbos-postgres, which has no consumer), refuse-completion-model (every executor lane but dbos-postgres, and both host / client backends) and hold-entry-commit (the JS, Temporal and CF Workflows executor lanes), plus request-stop (every executor lane, driver.requestStop) and, on cfw-workflows-d1 only, fault-stream-info (driverStreamFaults(driver)), fault-entry-history-read (driverHistoryFaults(driver)), observe-workflow-instances (driverWorkflowInstances(driver)) and terminate-workflow-instance (driver.terminateRunOwner); see the timing-controls table. attach · send · wake · evict · reload · hold-host-hook · hold-entry-hook · hold-tool-gate · hold-agent-hook are declared (Phase B; hold-entry-hook by WA MR1 Task 20), each with an optional driver member: driver.attach(agent, sid, opts?), driver.send(agent, sid, message, opts?), driver.evict(), driver.wake(sid, opts?), driver.reload(sid), driver.hostHooks, driver.entryHooks, driver.toolGates and driver.agentHooks. Reach the operations through driverOp(driver, op), the holds through driverHostHooks(driver) / driverEntryHooks(driver) / driverAgentHooks(driver) and the gates through driverToolGates(driver). The executor driver has hold-tool-gate and observe-tool-witness (driver.toolGates / driver.toolWitnesses, in-process ports; see "Scenario agents" above), on the JS lanes only hold-run-start-commit (driver.runStartHolds, see "Run-start commit holds" under "Timing controls"), and none of the others. Each executor lane's exact set is in drivers/capabilities.ts (EXECUTOR_JS_CAPABILITIES, EXECUTOR_TEMPORAL_CAPABILITIES, EXECUTOR_CFW_CAPABILITIES, and the base EXECUTOR_CAPABILITIES, which is dbos-postgres's). The host driver (below) has attach/send and hold-agent-hook on both hosts, and the other timing controls (evict, wake, hold-host-hook, hold-entry-hook, hold-tool-gate; see "Timing controls" below) on cf-do only. submit-approval has no driver yet. The ScriptedModel's own holds (hold, holdAfterDeltas) are not driver capabilities: they work on every lane (see "Timing controls").

Hard abort and reconnect (abort, reconnect). Both are executor-tier capabilities, on every executor lane (js-*, temporal-*, dbos-postgres, cfw-workflows-d1). The host and client tiers have neither, so a scenario that needs one is a declared gap there. discipline.abort-result-detail (scenarios/abort-detail.ts) uses both.

  • abort: requireAbort(driver, handle)(reason?) → Promise<StopResult>.
    • It calls the product handle's own abort(reason), never the soft interrupt that stop() sends. The default reason is 'conformance-abort'.
    • Per lane, that is: in-process on JS; a workflow cancel on Temporal; cancelWorkflow on DBOS; abortSession on CF Workflows, which terminates the owner instance and then fails the run framework_cancelled.
    • It targets exactly what stop() targets. A plain run is aborted itself. A client-tool chain is HALTED first, then its current run is aborted, with the same woken-run fallback.
    • The product's answer is reported, never thrown: { ok: false, error } when the target's handle was never obtained or its abort() threw (error is that throw).
    • A HarnessError is rethrown. requireAbort itself throws one when the scenario did not declare 'abort' in needs, or when the driver claims 'abort' but returned a handle without it.
    • Read the aborted run's terminal state as usual, through the handle's outcome() and observe-persisted.
  • reconnect: driverOp(driver, 'reconnect')(agent, sessionId) → Promise<RunHandle>.
    • It is the executor's own getHandle(agent, sessionId): what a caller that lost its handle, or never had one, does. The handle is wrapped and tracked like any run, so the RunLog sees it and its stop() / abort() reach it. It has no client-tool responder.
    • Its outcome() is that handle's result(), verbatim.
    • The product's answers are outcomes, never a HarnessError. A getHandle that answers null is { terminal: 'none', source: 'getHandle:null' } (NO_RUN_TO_RECONNECT_SOURCE), and stop() / abort() on that handle report { ok: false } (there is nothing to end). A getHandle throw is terminal: 'threw'.
    • A HarnessError rejects outcome() and is recorded on the RunLog. An un-prepared agent or a disposing driver is refused with one.
    • It is not attach: attach is the host and client tiers' chat-route resume, whose options are the resume headers.

Session delete (delete-session). An executor-tier capability, on every executor lane. session.delete-cascade (scenarios/session-delete.ts, RM-64) uses it.

  • driverOp(driver, 'deleteSession')(sessionId) → Promise<DeleteSessionResult>.
  • It calls the lane's REAL store, BackendEnv.stateStore.deleteSession: the same object the runtime writes through (memory, Redis or Postgres on the Node lanes, under every runtime; on cfw-workflows-d1 the Node-side D1StateStore over the worker's own D1 database). The harness deletes nothing itself, so the cascade to child sessions (RM-64) is the store's.
  • The product's answer is reported, never thrown: { ok: true }, or { ok: false, error } with resolveErrorDetail of the store's rejection.
  • A HarnessError is thrown when the driver is being disposed, when a run the driver started on that session has not finished (settle its outcome() first), or when the store itself raised one.
  • Read what the delete left through observe(), which is store-direct: a deleted session's persisted read is { ok: false } with a SessionAbsentError (reason: 'not-in-store'), and its runRecords and subSessions are [].
  • No host or client lane has it. cf-do's store is each session's DO-SQLite, whose deleteSession refuses (typed framework_not_supported), and the harness reaches it only through a read-only adapter. No scenario at those tiers needs it, so gaps.ts lists no gap for it.

Observation channels are capabilities too. Each Observation channel belongs to one observe-* capability: observe-persisted (persisted and subSessions), observe-llm (llm), observe-hooks (hooks), observe-stream (stream, host tier), observe-client (client, client tier). Put in needs every channel your scenario reads. runCell hands the scenario a per-cell driver view (drivers/cell-driver.ts) that enforces both sides at observe time:

  • reading a channel NOT in needs throws a HarnessError (a :crash) on every lane, the executor ones included — so a forgotten need is caught where the scenario runs today, never as a vacuous ?? default pass on a tier that lacks the channel;
  • a driver whose observation lacks a channel it claims, or carries an optional channel it does not claim, is a HarnessError (its static capabilities are wrong).

With the need declared, a lane whose driver cannot produce the channel records a declared gap and never runs. The executor driver declares observe-persisted, observe-llm, observe-hooks, observe-chunks and observe-run-records. It also declares observe-client-tool-submits, which is not a channel: every handle it returns reports when its client-tool responder began submitting each answer (see "Prove your window" below).

Observation (observation/observation.ts) fields a scenario can read via driver.observe(sessionId):

  • persisted (PersistedRead result union — see requirePersisted): status, error/errorDetail, messages/messageCount, customState, output (the session's stored structured output, verbatim), checkpoints (every checkpoint via listCheckpoints, in whatever order the store returns — select by id, never rely on array order; each carries getCheckpoint's stepCount, messageCount, streamSequence and timestamp, verbatim), latestCheckpointId (a SEPARATE, non-atomic read of the most-recently WRITTEN checkpoint — may not appear in checkpoints), listedMessageCount ({supported:false} on a store that can't listSessions, e.g. the DO — never inferred, only declared per backend; a scenario that asserts it needs the 'observe-listed-count' driver capability, so a non-listing backend is a declared gap, and reads it with requireListedCount(p). The capability is enforced against the backend at two points, each a HarnessError: buildBackendInstance refuses a lane whose driver advertises it over persistedReadOptions.listSessions: false, and the executor driver's observe() refuses a successful read that carries no listed count), version, pendingClientToolCallIds, clientToolCallOwnershipIds.
  • llm — this session's ScriptedCallRecord[] (model.callsFor).
  • stream? (StreamObservation, observation/stream.ts, host tier) — the raw chat-route SSE frames (seq from id:, event, data, at ms after the request), exchange (this exchange's 1-based position among the session's exchanges through the driver: 1 = the start's request, 2 = the first auto-send, send or attach), ttfbMs, ttfFirstNonHeartbeatMs (heartbeats and the [DONE] sentinel excluded), the response's httpStatus (and httpBody, verbatim, when it was not an SSE stream — e.g. a 409 JSON), open: true on a snapshot of a still-open connection, and terminal: 'none' whenever the connection closed without a data: [DONE] frame (never inferred as ended, even after a finish event), else 'failed' (an error event), 'interrupted' / 'paused' (data-run-interrupted / data-run-paused not reopened by data-run-resumed), or 'ended' (which does not imply a finish event — read frames). Malformed frame data is an ObservationError.
  • client? (ClientObservation, observation/client.ts, client tier) — what ONE mounted instance of the product client showed: instance (1 = the first mount, 2 = after one reload / attach), the UI events its transport read, the final status / error, helixError (the product's own parseHelixChatError(chat.error): the chat route's typed { error?, code, retryable?, cause? } body when the Chat's error carries one, absent otherwise; scenarios cannot import ai-sdk, so the driver runs the parser for them), every onError and onFinish call, every HTTP request it made (http: method, path, request headers and body, the response status, or failed when no response came), and the final UI messages. See the client tier below.
  • hooks? (HookObservation, observe-hooks) — suspended/resumed/ completed counts, failed (ErrorDetail[] in firing order), messages (one entry per onMessage firing: messageIndex, role, text, toolCallId?), and host? — the HOST's own hook firings ({ hook, status }: 'host.beforeEntry' for every /start admission, with no status, and 'host.onRunSettled' with the settled status; filter by hook when you judge one), present only where the host has recordable host hooks (cf-do); absent on the executor tier and the JS contrast host.
  • chunks? (ChunkObservation, observe-chunks, executor tier) — the session's stored stream, read through the backend's own StreamManager: chunks (getAllChunks, verbatim), info and status. info (ChunkStreamInfo | null) is ONE getStreamInfo(sessionId) read, projected verbatim: status, latestSequence, and metadata with only the keys in STREAM_INFO_METADATA_KEYS (error, errorDetail, finalOutput, currentRunId) whose value is not undefined. Values are the manager's own, never cloned, parsed or defaulted, so an absent errorDetail stays absent. null means the manager has no stream. On cfw-workflows-d1 the manager is the binding-side DurableObjectStreamManager; elsewhere it is the memory, Redis or Postgres manager. status is derived (info?.status ?? null), never read separately. A manager throw, or an info whose metadata is not an object, is an ObservationError. No new capability: info rides on observe-chunks. stream.fail-detail-on-info reads it.
  • runRecords? (observe-run-records, executor tier) — the session's run records as listRuns returns them (every page): runId, turn, status, verbatim. Count runs against a pre-entry snapshot with it (a loser must write no run record), and find a run's status by its id.
  • stopRecords? (observe-stop-records, executor tier, every executor lane) — the session's stop records: readStop for each run listRuns lists, in run-record order, each as runId, kind, reason, verbatim; a run with no stop record is absent. A failed readStop is an ObservationError. Read it beside runRecords to tell a recorded stop on a run that is still running (its owner signalled, its terminal not yet written) from a stop whose terminal already landed: the window proof of stop.drain-lapse-after-stop (CP-197).
  • subSessions — every SubSessionRef projection for this session (subSessionId, agentType, parentToolCallId, status); REQUIRED, [] when none spawned — never ?? [] a missing driver read.

BackendEnv declarations a driver author (never a scenario) reads: persistedReadOptions.listSessions, clientToolContinuation ('resume' | 'auto'), clientToolSubmitWakesRun (when true, the responder waits for the woken run to clear its pending entry before it resumes, and — if that resume() throws — for the woken run to leave 'active' before the chain settles, so it never outlives the cell — CP-64), the optional terminateCellWork hook (Temporal, and the CF Workflows lane: ends a cell's whole workflow tree at cell end and judges "stopped" from the backend's execution status — see §3, "A cell never leaks running work" and "The CF Workflows lane"), and the optional portControls (the CF Workflows lane supplies bridge-backed tool gates and witnesses in place of the driver's in-process ports).

The host tier (drivers/host-driver.ts) ​

The host driver drives the product's chat routes over raw HTTP — no React, no ai client drives the run — on two backends: cf-do (the conformance worker's routes, a line-cited mirror of examples/opennext-cloudflare-do/src/app/api/chat/[sessionId]/route.ts, in front of the real DurableObjectAgentBase subclass) and js-chat-host (harness/setup-helpers/js-chat-host.ts: handleChatStream over js-memory served by a Node http server, the REAL request passed through so the handler reads its own resume headers). Both expose POST/GET /agents/:agentType/chat/:sessionId and POST /do/:sessionId/interrupt.

Scope: the host tier tests the SERVER contract; the client tier tests the product client. Every host-tier decision (the outcome, and what and when the responder answers) is made from the server's frames as the harness parser reads them. What the product client would do with the same bytes is recorded, never used to decide (clientRejected, below).

Primitives:

  • start(agent, message, sid, opts?) — a new client conversation: POST { id: sid, messages: [user], trigger: 'submit-message' }, exactly what ai's Chat.sendMessage sends through DefaultChatTransport + prepareHelixChatRequest. It returns once the request is dispatched; outcome() reads the SSE to its end.

  • send(agent, sid, message, opts?) — the same POST on the session's existing conversation (as assembled so far, a live stream's partial assistant included) plus the new user message.

  • attach(agent, sid, { resumeFromSequence?, existingMessageId? }) — GET with exactly those resume headers (none when not given).

  • stop(reason?) — POST /do/:sid/interrupt (cf-do: the DO's /interrupt; JS: agent-server's own interrupt route — the durable flag plus a bounded wait for the run to leave active). A non-2xx is REPORTED as { ok: false, error }, whose message names the HTTP status and body; a transport failure is a HarnessError.

  • agentHooks (both hosts) and evict() / wake(sid, opts?) / hostHooks / toolGates (cf-do only) — the timing controls (see "Timing controls" below); a body an evict() severs is none / stream:severed-by-evict.

  • driveDoFrontendHandle(agent, sid, { input, leg }) (capability drive-do-frontend-handle, cf-do only, host tier) — drives the runtime's own DOFrontendExecutor inside the cf-do Worker (/__conformance/do-frontend-stream): execute(...) with the bundled agent, then iterating the handle's stream({ fromSequence: 0 }). It returns { attached, result }. attached resolves with { viaChunk }: true once the stream yielded its first chunk (a LIVE reader; leg: 'before' opens the stream at once, so a scenario holding the model call releases it after a viaChunk attach), false when it ended, threw or was null before any. leg: 'after' first waits (bounded) for the run's terminal /status AND the DO's stream to read terminal (K5 → run status → failStream). result is a DoHandleStreamRead: outcome (ended / threw / null), every chunk's type (an error chunk with its errorDetail), the throw, streamStatusAtOpen, and the DO's /status (status, persistedErrorDetail, the oracle). A failed drive rejects both promises with a HarnessError. The run counts as one the driver ran; the cell-end reap cancels a read that never ends. js-chat-host has no DO executor (a declared gap). DI-43 (discipline.do-handle-failed-stream).

  • streamInfo(sid) (capability observe-stream-info, cf-do only) — driverOp(driver, 'streamInfo')(sid) → Promise<HostStreamInfo>. It reads the session's stream info through the host's two stream managers:

    • client is the manager the chat routes use (DOStreamManagerClient, over the DO's /snapshot wire);
    • inProcess is the DO's own DOStreamManager.

    Each read (StreamInfoRead) is either { ok: true, value }, with value the ChunkStreamInfo projection that observe-chunks' info uses (or null when the manager answers "no stream"), or { ok: false, httpStatus, body } when the manager THREW. That throw is the product's answer, reported verbatim. A transport failure or a malformed reply (including a foreign sessionId, or ok disagreeing with the HTTP status) is an ObservationError. Reading a session this driver never ran is a HarnessError.

    The two reads are separate requests made one after the other, not one atomic read, so read a settled stream. Each one WAKES an evicted DO: never call it between evict() and wake(). A client value of null can also mean the DO's /snapshot answered non-2xx (DOStreamManagerClient maps any non-ok answer to null); read the snapshot as well to tell the two apart.

    js-chat-host does not have it (no Durable Object stream managers), and neither does the client tier. stream.fail-detail-on-info-host declares the js-chat-host gap in gaps.ts; the executor-tier stream.fail-detail-on-info cell is its in-memory contrast.

  • disconnect(sid) (capability client-disconnect-at-write, both hosts; CP-102) — the client goes away mid-stream: the driver aborts the session's latest open exchange's own fetch and cancels its body, as a browser reload or a closed tab would. The cut exchange ends none / stream:client-disconnected with the frames read so far — a driver-initiated close, never a product stream:no-done, and distinct from dispose's abort (which stays a HarnessError). No open exchange is a HarnessError. It never touches the run: read the run's outcome afterwards with attach or a snapshot. The capability promises only that the host learns of the cut at the latest at the response's NEXT WRITE: a scenario asserting the cut lands while the response is quiet also needs client-disconnect, which has NO member: it only means the host learns of a cut while the response is quiet (js-chat-host only). cf-do lacks it (a declared gap): Miniflare, like wrangler dev, reports a client disconnect to the Worker only at the response's next write. Cloudflare's edge reports it in within about 0.3 s with enable_request_signal (CP-102 edge probe, 2026-10-08), which the local lanes cannot show.

  • openUpstreamReaders(sid) (capability observe-upstream-readers, both hosts; CP-102) — a PRODUCT answer: how many of the host's own product readers of the session's stream are open. js-chat-host: the readers the env's one stream manager handed out and has not closed (the instance is patched in place, so the executor's reader for the chat route is counted). cf-do: the worker's open upstream /sse bodies — the chat handler's DO namespace is wrapped at its seam (countingNamespace, harness/cf-do/worker/upstream-reader-counter.ts), each /sse body a pull-based pass-through counted until the product cancels it or reads it to its end — read through GET /__conformance/worker-upstream-readers/:sid. Worker-isolate state: it never reaches a DO, so it is safe between evict() and wake(). Pair a "closed" assertion with a >= 1 assertion taken while the stream is live, or it is vacuous.

Outcome mapping — only from what the caller observed; the raw signal is in source, nothing is synthesized:

Observedterminalsource
non-2xx / 204 responsenonehttp:<status> (error = body verbatim)
SSE closed with no [DONE]nonestream:no-done
error event, then [DONE]failedstream:error (error = its errorText)
data-run-paused, then [DONE]suspendedstream:data-run-paused
data-run-interrupted, then [DONE]interruptedstream:data-run-interrupted
finish, then [DONE]completedstream:finish
only data-resume-rejected events, then [DONE]nonestream:data-resume-rejected (error = the frames)
[DONE] alonenonestream:empty
the driver's own disconnect() cut itnonestream:client-disconnected (a driver-initiated close)
anything else—ObservationError (a crash, never a guess)

A transport failure is a HarnessError; a 2xx that is not text/event-stream is an ObservationError; a stream the driver aborted itself (dispose) is a HarnessError. observe().stream is the session's LATEST exchange through the driver (an open one as an open: true snapshot); observing a session with no exchange is a HarnessError.

The responder (clientTools) answers with exactly the auto-send the product client would POST. It keeps each session's conversation the way ai's Chat would, built from the SERVER's frames: the harness parser's events are fed to ai's readUIMessageStream, with no client-side schema gate. Frames that cannot be applied at all (a server contract break) make the responder an ObservationError, never a silent suspended. When an exchange ends paused, the responder answers every input-available client tool part of the last assistant message (useResumeClientTools' dispatch filter). The pending calls come from the server's tool-input-available frames, because the data-run-paused frame names no call ids today.

  • at: 'paused-end' — delayMs after the PAUSED TERMINAL FRAME (the [DONE] ending the paused stream) arrived;
  • at: 'input' — delayMs after that call's tool-input-available frame, never before the exchange ended.

Answers that come due while a request is in flight are applied when it ends and ride ONE auto-send (Chat's tail check), exactly as ai's Chat batches them — never one POST per answer.

Answering is Chat.addToolOutput's part update plus, when the PRODUCT's createHelixSendAutomaticallyWhen predicate fires (imported from its React-free source module, one instance per session), the AUTO-SEND: a POST whose body is { id, messages, trigger: 'submit-message', messageId }, with messages the whole conversation and messageId the last message's id. The route hands that to the handler's resume path. The auto-send's response becomes the next exchange, and outcome() is the last exchange's. Like useResumeClientTools, every call whose tool is configured is answered and a call whose tool is not is left pending (the executor tier's stricter "block the whole round" rule is the executor driver's only — it has no product client to mirror). A queued answer whose call is no longer on the last assistant message when its time comes is an ObservationError, never a dropped answer. stop() halts the chain. The whole chain is ONE tracked run: a chain that outlives its cell fails the leftover-run check and dispose().

On js-chat-host, stop() goes through agent-server's interrupt route (a durable flag the JS loop polls between iterations). It waits at most JS_INTERRUPT_OBSERVATION_DEADLINE_MS (10 s) for the run to observe it, so release any held model call or tool within that bound. Past it, the stop is a HarnessError (the host marks the answer with x-conformance-harness-error and fails endCell), never a product { ok: false }.

clientRejected is an observation. The response body is teed, and the second branch runs through exactly the product client's pipeline (DefaultChatTransport's strict uiMessageChunkSchema into readUIMessageStream). Whatever that pipeline rejects is recorded, except the server's own error events. It lands on the exchange (observe().stream.clientRejected) and on the run's outcome (RunOutcome.clientRejected, as #<n>: <ai's message> for each rejected exchange of the chain). It never gates the answer. Pin it under its own assertion id, for example <scenario>.client-accepts-stream. Until DI-20's fix (!268) every run on both hosts recorded one: the server's finish-step frame carried finishReason, and ai 6.0.49 rejected it. Since the fix the step frames are bare { type }, and nothing is recorded.

The client tier (drivers/client-driver.ts) ​

The client driver runs the product's REAL React hooks under react-test-renderer (no DOM): useHelixChat (the @ai-sdk/react useChat plus useResumeClientTools plus the Helix sendAutomaticallyWhen), over the real DefaultChatTransport with the real prepareHelixChatRequest / prepareHelixReconnectRequest. The component (drivers/client-harness.tsx) is a line-cited mirror of the opennext example's app/chat/[sessionId]/ChatClient.tsx, mounted from the snapshot its server page loads. Its deviations are listed in its header: the transport's fetch is routed, the route layout is the conformance host's, onError / onFinish are recorded, and useAutoResync is not mounted. It runs on the same two backends as the host tier, cf-do and js-chat-host.

Scope: the client tier records what the product client DOES, faithfully, including what it gets wrong. The only harness piece between the Chat and the server is backendFetch, the transport's fetch. It routes each request to the backend, records it in client.http, and passes every response byte through unchanged while recording the SSE frames the transport reads in client.events. There is no schema patch, no frame stripping and no custom transport. There is no observe-stream at this tier: the client sees UI events, not raw frames.

The step-event rejection (DI-20, fixed). Both hosts use includeStepEvents: true, like the example. Until !268 the server's finish-step frame carried finishReason, which ai 6.0.49's strict uiMessageChunkSchema refuses. So the real Chat ended EVERY response with status 'error' ("Type validation failed …"). In a client-tool round the step's finish-step comes before the tool part on the wire, so no handler ran and nothing was auto-sent. The client tier reported exactly that: failed with source: 'chat:error'.

Since the fix the step frames are bare { type }. The Chat completes, and the client-tool round is answered and auto-sent. The live smokes (harness/cf-do/__tests__/client-driver.integ.test.ts) now pin that behaviour. This is the worked example of a client-tier finding: the driver recorded the product's failure faithfully, and the fix flipped the cells without any harness change.

Each mount of a session's client is one instance. Every instance has its own page network and its own records, so a torn-down instance's late callbacks never land in its successor's observation. observe() reports the session's latest instance.

Primitives:

  • start(agent, message, sid, opts?) — the example's page load plus the first message. It GETs the snapshot route (a non-2xx gives the page's default 'ended' snapshot, as in page.tsx), mounts the ChatClient from it, and calls the hook's sendMessage({ text }). A session that already has a mounted instance is a HarnessError; use send instead.
  • send(agent, sid, message, opts?) — the mounted instance's sendMessage({ text }), the hook API. The example's input form refuses while streaming; the hook does not.
  • attach(agent, sid, { resumeFromSequence?, existingMessageId? }) — a remount with resume: true and exactly those positions, carrying the previous instance's messages (or none). useChat's mount effect sends the reconnect GET.
  • reload(sid) — the consumer's browser reload, mirroring the example exactly:
    1. Unmount the instance. useResumeClientTools' cleanup aborts its in-flight handlers, so an aborted handler never submits.
    2. Drop its page network. Every open request and body of the old instance is aborted, and any later request it tries fails without reaching the server.
    3. Load the snapshot as the page does (snapshot-load.ts): a 404 is the empty new-session snapshot, any other failure is retried (250 / 500 / 1000 ms), and after the 4th the page's error state is a HarnessError.
    4. Mount a NEW instance from it as use-snapshot-chat.ts does: the product's snapshotResumeOptions(snapshot) resumes only while the snapshot is live (active / paused), from its streamSequence, under its resumeMessageId with the replayed prefix (X-Helix-Replay-Prefix), seeded without that message; and useHelixChat's resync (the page's snapshot, the routed fetch) re-anchors on a resync trigger. The triggers it acted on are the observation's resyncs.
  • stop() — chat.stop(). It aborts only the Chat's own fetch; whatever the server does about it is observed (persisted, llm), never assumed.
  • agentHooks (both hosts) and evict() / wake(sid, opts?) / hostHooks / toolGates (cf-do only) — the timing controls (see "Timing controls" below). A body an evict() severs is the Chat's own error, not a harness failure.

The responder (clientTools) is the instance's toolHandlers. The real useResumeClientTools dispatches every input-available call of a configured tool; the handler waits for its at, then delayMs, then returns output. The product's own addToolOutput and sendAutomaticallyWhen make the POST.

  • at: 'input' — counts from the dispatch (the tool input reached the Chat);
  • at: 'paused-end' — waits until the Chat's status leaves 'streaming' / 'submitted' after the tool part arrived: an observable condition, never a sleep, bounded by CLIENT_PAUSED_END_BOUND_MS (a HarnessError past it, and the handler then never answers).

Outcome mapping, once the instance is quiescent (the op settled, the Chat is not 'submitted' / 'streaming', no request or body is open, no handler is in flight, and no returned answer still waits to be applied, confirmed across a React flush). It comes only from the Chat state, with the raw signal in source:

Observedterminalsource
the instance was torn down first (reload / attach / dispose)noneclient:unmounted
Chat status 'error'failedchat:error (:disconnect per onFinish); error = the Chat's message
'ready', the last onFinish had isAbortnonechat:ready:abort
'ready', no chat-route requestnonechat:ready:no-request
'ready', the last request a reconnect answered 204nonechat:ready:reconnect-204
'ready', the last request's events: data-run-paused (not reopened)suspendedchat:ready:data-run-paused
… data-run-interruptedinterruptedchat:ready:data-run-interrupted
… a finishcompletedchat:ready:finish
… only data-resume-rejectednonechat:ready:data-resume-rejected
… nothing / [DONE] alonenonechat:ready:empty
… anything else—ObservationError
the hook's sendMessage threwthrewchat:sendMessage-threw

The Chat's 'error' covers everything it surfaces as one: a client-side schema rejection, a non-2xx body, a server error event's text, or a severed body. Read error and client to tell them apart.

A harness failure on the instance makes outcome() reject with a HarnessError and makes observe() throw. Such failures include a backendFetch transport failure that neither the Chat, a reload nor evict() caused, a malformed frame, and the paused-end bound. Every run op and every reload's new instance is one tracked run. dispose() unmounts every instance, drops its connections and drains, bounded by CLIENT_DRIVER_DRAIN_BOUND_MS.

The store tier (store-suite.ts) ​

A store-tier scenario exercises a SessionStateStore directly, with no agent, runtime or stream: it exists so a store-level finding (RM-26 / RM-27 / RM-53 / RM-54, and later RM-13 / RM-15 and DI-11's usage-store clause) can be provenBy a cell on every store backend, not only by a store package's own vitest suite.

  • Lanes. harness/store-backends.ts (STORE_BACKENDS): store-memory, store-redis, store-postgres, store-d1 (D1StateStore over a Miniflare D1 binding) and store-do-sqlite (DOStateStore over node:sqlite, NOT workerd's SQLite). Each backend sets up a fresh store per lane and names its static capabilities (STORE_*_CAPABILITIES in drivers/capabilities.ts), which must equal its lane's in CONFORMANCE_LANES (storeLaneProblems; the suite refuses to run otherwise).
  • The driver. makeStoreDriver builds a tier: 'store' driver whose prepare / start / observe throw a HarnessError (a store scenario runs no agent) and which records no runs. A scenario reaches the store through driverStore(driver) (needs 'store-direct'): store, sessionStore() (a store for ONE new session: the lane's own store, except on DO-SQLite, where a store IS one session, so each call builds a fresh one) and, where the lane has one, the RM-53 read seam interleaveListCheckpoints.
  • Capabilities. 'store-direct' (every lane), 'clone-session' (not DO-SQLite: DOStateStore refuses cloneSession) and 'interleave-list-checkpoints' (Redis and D1; Postgres lists in one statement and memory / DO-SQLite synchronously, so they have no seam). A scenario that needs a missing one is a declared gap, as at every tier.
  • Contract cases, not copies. Each store id runs ONE named case of core's store contract (@helix-agents/core/testing: commitCheckpointScenarios, checkpointListingCases), looked up by its title prefix; zero or several matches is a HarnessError, never a pass. The store packages' own suites and these cells run the same code, so a contract fix is proven on every lane at once. To add one: export the case from core's testing module (keep its title), add a scenario in scenarios/store.ts that names it, and declare any gaps.
  • Self-tests. src/conformance/__tests__/store-tier.test.ts (in test:conformance-unit) checks the driver's refusals, the lane/backend agreement, and runs every store scenario the memory lane can run against the in-memory store, so a renamed contract title fails there first.

Timing controls (drivers/timing-controls.ts) ​

The races and wake bugs this suite chases need the harness to hold the product at a precise point. On cf-do the host AND client drivers expose five timing controls, plus one probe of the host's live state; js-chat-host's drivers expose one of them (hold-agent-hook). The ScriptedModel adds a sixth, holdAfterDeltas, on every lane. Every control works through the product (a real hook, tool, request, alarm handler or model stream); none marks state directly. On the JS contrast lanes hold-host-hook, hold-entry-hook, hold-tool-gate, evict, wake and observe-host-status are declared gaps (drivers/capabilities.ts says why). The executor tier has two of the driver controls, hold-tool-gate and observe-tool-witness, backed by in-process ports (harness/in-process-ports.ts), and on the JS lanes a third, hold-run-start-commit (run-start commit holds, harness/run-start-holds.ts); it has no host hooks, evict or wake.

CapabilityMemberWhat it holds or does
hold-host-hookdriverHostHooks(driver)the worker's hooks.onRunSettled for a session blocks inside its body while held. The DO delivers it AFTER releasing its execution (WA MR1), so a held one never keeps the DO executing
hold-entry-hookdriverEntryHooks(driver)the worker's hooks.beforeEntry (admission) for a /start of a session whose user text is the held message blocks while held, holding the DO's producer claim, so a second entry is refused as already running
hold-agent-hookdriverAgentHooks(driver)the AGENT's onAgentSuspended for a session blocks inside the harness's hook wrapper while held (cf-do and js-chat-host)
hold-tool-gatedriverToolGates(driver)a bundled tool that calls its gate blocks inside execute while held (cf-do: the bridge; executor tier: in-process ports)
hold-child-phasedriverChildPhaseHolds(driver)CP-106: arm(sid, point, { toolCallId? }) holds the session's next child-phase fire at point (afterCommit: after the step commit, before a child call's ref write; afterRefWrite: after the ref write, before the child starts, only in the attempt that writes the ref; afterChildStart: after the child started or was re-attached, before the start fence); whenHeld resolves with the fire's { sessionId, runId, toolCallId } (the observed event to anchor an eviction or a stop on), release continues it, and the cell end releases every hold. Through the product's test-only childPhaseHooks only: the runtime-js executor's on the JS executor lanes and js-chat-host, the conformance DO's DurableObjectAgentConfig.childPhaseHooks (a bridge gate, answered when its isolate goes away) on cf-do. Companion calls (MR A); sub-agent calls with MR B
evictdriverOp(driver, 'evict')()disposes workerd and recreates it over the same DO-SQLite (module state gone, storage kept)
wakedriverOp(driver, 'wake')(sid, { via })wakes an evicted DO through a real product request or its real alarm handler
observe-host-statusdriverOp(driver, 'hostStatus')(sid)reads the host's live state: { isExecuting, sessionStatus } (the DO's isExecuting); a DO fetch, so it WAKES an evicted DO: never call it between evict() and wake()
hold-run-start-commitdriverRunStartHolds(driver)executor tier, JS lanes only: an entry's run-start commit (appendMessages with startRun) blocks before it runs, or right after it lands, keyed by session and entry
fault-stream-infodriverStreamFaults(driver)executor tier, cfw-workflows-d1 only: arm(sid) makes the session's NEXT StreamServer /info read answer 500 once (the workflow body's stream namespace is wrapped in the worker); waitUntilServed proves the read happened; an arm never served fails endCell (DI-21). arm(sid, { persistent: true }) fails EVERY /info read of the session until disarm(sid) (DI-46: a failure that outlasts the engine's step retries)
fault-entry-history-readdriverHistoryFaults(driver)executor tier, cfw-workflows-d1 only: while armed (arm(sid) … disarm(sid)), the workflow body's getMessageCount for the session reads one row too many, so the entry plan's loadLLMHistory throws state_history_incomplete BEFORE its run-start commit; waitUntilServed proves a faulted read happened; an arm never served fails endCell; disarm before judging, so the cell's own reads see the real count (CP-100)
observe-workflow-instancesdriverWorkflowInstances(driver).list(sid)executor tier, cfw-workflows-d1 only: the session's workflow instances, each with its engine status read straight from the binding (DI-34, CP-15)
terminate-workflow-instancedriverOp(driver, 'terminateRunOwner')(sid)executor tier, cfw-workflows-d1 only: terminates the instance owning the session's current run through the raw binding, as the dashboard or wrangler would, and waits for the engine to report it terminated; the ledger marks it terminatedBy: 'external', so the cell's reap neither fences nor re-terminates it (DI-34)
request-stopdriverOp(driver, 'requestStop')(sid, r)executor tier, every lane: records an interrupt stop for the session's CURRENT run directly in the store (WA MR2's per-run StopRequest, record only, no deadline: nothing is signalled or settled by the call), the cross-process stop every runtime reads; returns the store's RequestStopResult verbatim (no_live_run is a product answer) (CP-106, CP-108). Observed back through stopRecorded on the persisted channel
stop-answerdriverOp(driver, 'stopAnswer')(sid, req)host tier, cf-do only: calls a stop surface (via: 'route', the DO's /interrupt or /abort; via: 'endpoint', EndpointContext.stop through the worker DO's production custom endpoint STOP_ENDPOINT_PATH) and returns its RAW answer (HTTP status, parsed body, answer time); never waits for the run. A request cut by a later evict() is severed. WA MR2's D-1 / D2 answers (owner batch 21: CP-14, CP-112, CP-113, CP-114)
hold-entry-commitdriverRunStartHolds(driver)the same hold (and waitUntilSettled) on the JS, Temporal and CF Workflows executor lanes, WITHOUT the promise that a concurrent entry reaches its commit
provider-connection-dropdriverProviderDrops(driver)executor tier (every lane) and host tier (both hosts): the reserved provider-drop agent runs on the REAL VercelAIAdapter over the instance's provider mock, on the lane's own fetch; script(cellId, drop, then) closes or resets the cell's next request(s) before any byte, after the headers or after N deltas, requests(cellId) is the inside probe
client-disconnect-at-writedriverOp(driver, 'disconnect')(sid)host tier, both hosts: aborts the session's latest open exchange's fetch mid-stream (a reload / closed tab); the host learns of it at the latest at the response's next write; HarnessError when no exchange is open or the cut did not land
client-disconnect— (no member)host tier, js-chat-host only: the host learns of a cut PROMPTLY, while the response is quiet (cf-do: Miniflare reports only at the next write, a declared gap)
observe-upstream-readersdriverOp(driver, 'openUpstreamReaders')(sid)host tier, both hosts: a product count of the host's own open readers of the stream (cf-do: the worker's open upstream /sse bodies, countingNamespace; JS: the stream manager's open readers)
hold-stream-claimdriverStreamClaimHolds(driver)host tier, both hosts: while held, a chat entry for the session blocks right before its claimStream, AFTER its run-start commit (the run is current and live, a snapshot reports it active, and its stream still shows the previous turn's status: the run-start window). cf-do: the bridge's beforeEntry reply arms the DOAB subclass's claim gate for a /start admitted while held (host.streamClaim.gate), so an unarmed entry adds no call or await; js-chat-host: the stream manager's claimStream is gated in place (harness/stream-claim-gates.ts). RM-73 (resume.run-start-window, stream.info-run-start-window)
hold-run-enddriverRunEndHolds(driver)host tier, both hosts: while held, the host's run for the session blocks right AFTER its stream's terminal write (endStream or pauseStream has completed, so a client already sees the finish / pause) and BEFORE the host releases the run. cf-do: the bridge's beforeEntry reply arms the DOAB subclass's run-end gate for the run of a /start admitted while held (host.runEnd.gate, after the DO stream manager's own call, before the drive's handle.result() settles and enterTail runs), so an unarmed entry adds no call or await; js-chat-host: the stream manager's endStream / pauseStream are gated in place after their own call, before the executor's result resolves (harness/run-end-gates.ts). CP-133
fault-do-submit-answerdriverSubmitFaults(driver)host tier, cf-do only: arm(sid, { status, body }) makes the session's NEXT client-tool submit to the DO be answered with that status and body before it reaches the DO (the worker wraps the chat handler's DO namespace, harness/cf-do/worker/submit-fault-namespace.ts, as a crashed isolate or a proxy's error page would answer); one-shot; waitUntilServed proves the submit was answered; an arm never served fails endCell. js-chat-host's submit never crosses an HTTP boundary (a declared gap). DI-47 (submit.do-untyped-answer)
fault-agent-factorydriverFactoryFaults(driver)host tier, cf-do only: arm(sid, { target?, message, code?, retryable? }) makes the worker's agent resolver (or, with target: 'usageStore', its usage-store factory, which otherwise returns a store equivalent to DOAB's internal one) throw on every resolution for the session (every drive and every wake); with target: 'usageStoreWrites' (SDK-4) the usage store builds and each of its recordEntry writes throws instead, each counted in served: a typed HelixError when code is given, else a plain Error (classified transient). The fault lives in DO storage, so it survives evict(); disarm and served read it back. Every call is a DO request that WAKES an evicted DO: never between evict() and the scenario's own wake. MA-01, DI-01, DI-16 (wake.factory-fault-*), MA-06 (submit.paused-factory-fault), DI-11 (factory.usage-store-unavailable), SDK-4 (chat.usage-write-failure-drain-completes-host)
observe-factory-contextdriverFactoryContexts(driver)host tier, cf-do only: seedIdentity(sid, { userId?, metadata? }) makes the worker pass that identity to handleChat on the session's NEXT chat request only (one-shot, worker-isolate state: what an app passes for the request that starts a session); read(sid) returns every context the worker's agent resolver received for the session, in order (purpose, drive or reason, identity; kept in DO storage, so read is a DO request that wakes an evicted DO). MA-03 (factory.context-carries-identity)
concurrent-entry-policydriverOp(driver, 'concurrentEntry')(sid, p)host and client tiers, cf-do only: sets the session DO's concurrentEntry policy ('interrupt_previous' or 'reject'), which it applies to every later drive exactly as DOAB's class-wide config.concurrentEntry would (the drive host's one field it feeds). Kept in DO storage, so it survives evict(). Call it before the session's first entry; a DO request that WAKES an evicted DO. js-chat-host has no policy (a declared gap). CP-13's interrupt half (chat.keyed-send-interrupts-live-run, its keyless control, client.keyed-send-interrupts-live-run; D-2 MR 1.5)

The Task 16A capabilities' live self-tests run on every lane that declares them: harness/__tests__/capability-probes.integ.test.ts (js-* and temporal-*), capability-probes-dbos.integ.test.ts (dbos-postgres: the contrast that keeps refuse-completion-model undeclared, and both probes recorded as declared gaps), harness/cfw/__tests__/capability-probes.integ.test.ts (cfw-workflows-d1, incl. the zero-cost-unarmed check) and harness/cf-do/__tests__/completion-support.integ.test.ts (cf-do and js-chat-host, host and client: H1, H2 and client.helixError). A capability that fails its self-test on a lane is not declared there.

Provider connection drops (provider-connection-drop). The control tests the real LLM adapter, so it bypasses the ScriptedModel: calls of the reserved agent PROVIDER_DROP_AGENT_NAME (agents/provider-drop.ts, built with providerDropAgent()) run on VercelAIAdapter + @ai-sdk/openai against a hermetic OpenAI-compatible mock (harness/provider-mock.ts, one per backend instance on 127.0.0.1:0). The routing sits where the lane's fetch runs (harness/provider-routing-adapter.ts): the suite's ProviderRoutingScriptedModel on the Node lanes, each worker's adapter factory (routeProviderCalls, URL from the PROVIDER_MOCK_URL binding) on cf-do and cfw, so a drop surfaces as that runtime's own error (undici's TypeError, workerd's Network connection lost.).

  • Arm the drop before starting the session: driverProviderDrops(driver).script(cellId, drop, { text }), where drop is a ProviderDrop: { phase: 'before-response' | 'after-headers', close, times } or { phase: 'after-deltas', afterDeltas, close, times }, and close is 'close' (FIN) or 'reset' (RST; undici reports the two with different cause codes). The next times requests of sessions named ${cellId}-… drop; every later one streams text (word deltas, as scriptedTextDeltas).
  • A drop after the response started is deterministic: the mock waits until the client acknowledges what it read (the routing adapter's acknowledgingFetch: the headers, or the Nth SSE frame, read into the client; protocol in harness/provider-ack.ts), bounded by ackBoundMs, so a reset never discards bytes the client has not yet read off the socket, and the request's outcome.ack records whether the acknowledgement came (received / missed; not-needed before the response). A drop meant to prove the adapter's retry belongs after the headers: a drop strictly before the response is already retried on Node (the AI SDK wraps undici's fetch failed). Never add a sleep to "let the bytes leave".
  • Non-vacuity comes from requests(cellId) (each request's outcome: pending, served, or dropped with its phase, close and deltas written), never from observe-llm: a routed call never reaches the ScriptedModel, so its call records stay empty. Do not script ctx.model for the reserved agent.
  • The classification is read inside the call: each routed generateStep gets its own call id (in the provider URL path, so the mock attributes every request to its call) and reports what the adapter returned (ProviderCallReport, a union: returned with the result type, resolveErrorDetail of an error step and the text deltas passed on, or threw) as it returns. calls(cellId) gives each call's requests and report, independent of the run's outcome and of what the runtime persists.
  • Harness failures, reported at the cell's end: a request with no script, a scripted drop never exercised, a connection not dropped at the scripted point (or whose receipt the client never acknowledged), a routed call that never reported (or reported twice, or malformed), and a routed call that reached the bridge on a workerd lane (the bridge refuses it before any model sees it).
  • The routing adapter is bundled into both workers: keep it free of Node imports. Import shared constants from provider-ack.ts, never from the mock (provider-routing-adapter.test.ts bundles both workers and fails on a Node server import; a worker that cannot start gates every cf-do / cfw cell). A routed call on a worker with no PROVIDER_MOCK_URL throws inside the worker, which the runtime turns into a failed step, so it surfaces as the cell's "scripted drop never exercised" (the mock saw no request), not as its own error. No other agent or scenario may use the reserved name (agent-provenance.test.ts).

Prove your window. Every hold gives you an "inside" probe (isInside for hook holds, waitUntilEntered for tool gates, model.whenHeld for model holds). A scenario that relies on a window being open asserts it under its own id BEFORE it acts, and treats a window that never opened as a named precondition failure, never as "the race did not happen" (CP-02's .first-entry-held, MA-09's .recovered-by-the-wake). And prove the window exists in PRODUCTION, not only in the harness: cite the product's async gap a caller can land in. A hold that only lengthens synchronous product code tests a window no caller can reach (WA MR1 Task 20 retired race A and lifecycle.send-in-terminal-tail for exactly this). A probe says the hold is in effect in the harness; what the PRODUCT does meanwhile (the stream, the snapshot, /status) is observed, never assumed.

A window that closes when the responder answers is judged against the answer's own instant, never a proxy. The executor driver records, on the run's handle, performance.now() (monotonic) immediately before its client-tool responder calls the product's submit (RunHandle.clientToolSubmits, capability observe-client-tool-submits; reach it with requireClientToolSubmits(driver, handle)). A scenario whose reads must land before the answer takes performance.now() after its last read and requires it to be strictly earlier than that instant, judged once the chain settled (stream.suspension-marker's .window-open, via judgeReadsBeforeAnswer). When no answer was recorded, the responder never answered the call, and that is a judged failure naming the chain's outcome, never a crash: the run completed without suspending on the call, or it threw or ended with a status the responder does not answer. An unsettled chain fails it as a precondition. A window judged while the chain is still running (the next answer is many seconds away, and the scenario acts on the window before the chain settles) uses judgeWindowBeforeAnswer instead (submit.partial-batch's .window-open and its .execute-refused-pending window, submit.partial-batch-resume's .nothing-submitted-window-open and .window-open). There a missing record for the call PASSES. That is sound because the check reads the record after the window's work finished, and the driver pushes each record synchronously before the product's submit is called. So a call with no record yet can only be answered after the window. An answer recorded at or before the done instant fails the check. Never derive the close from a time taken before start plus the answer's delayMs minus a margin: that proxy charges the backend's start-to-suspension latency against the window, so a slow runner can fail the check while the window really was open (FU-CONF-PARTIAL-BATCH-T0-WINDOWS, the pattern both partial-batch scenarios used until owner batch 17). The host and client drivers don't record the instant, so a scenario that needs it is a declared gap on those tiers.

Host-hook holds. hold('onRunSettled', sid), release, isHeld, isInside and waitUntilEntered('onRunSettled', sid, boundMs). isHeld only says a hold is in effect; isInside says a caller is blocked in the hook right now (entered, not released, and its connection not closed by an eviction). The DO's settle path (runtime-cloudflare do/drive/settle.ts) finalizes the run's own stream, releases the execution registration, and only then delivers onRunSettled, at least once per (runId, status) (deduplicate on its deliveryId, ${runId}:${status}). So a held settle hook never keeps the DO executing: lifecycle.settle-hook-releases proves the next turn is served while it is held. The hook is recorded in hooks.host as { hook: 'host.onRunSettled', status }, where status is the run's settled status (completed, suspended_client_tool, ...).

Entry-admission holds (hold-entry-hook). driverEntryHooks(driver): hold(sid, message), release, isHeld, isInside and waitUntilEntered(sid, message, boundMs). While held, a /start for sid whose user text is message blocks inside the worker's beforeEntry (admission: after the drive's busy check, before it registers its execution). A DO request drive holds the DO's producer claim across admission (runDrive), so a second entry for the session meanwhile is refused as already running, although the session it read still says completed: CP-02's real window (lifecycle.send-during-entry-admission). Only /start entries are gated. Every admission is recorded in hooks.host as { hook: 'host.beforeEntry' } (payload { drive, message }).

Agent-hook holds (hold-agent-hook, timing control A).driverAgentHooks(driver) has the host-hook API with a hook parameter: hold('onAgentSuspended', sid), release, isHeld, isInside and waitUntilEntered('onAgentSuspended', sid, boundMs). Only onAgentSuspended is holdable today (HOLDABLE_AGENT_HOOKS, harness/agent-hook-gates.ts); another hook is added only once both hosts gate it and a self-test proves it. The hold is in the HARNESS's hook wrapper, never in the product: on cf-do the worker's wireBridgeAgentHooks reports the firing to the bridge (hook.fired) and awaits the reply, which the bridge withholds while the hook is held; on js-chat-host wireAgentHooks consults the host's in-process gate set. Either way the firing is recorded FIRST (observe().hooks.suspended counts it while held), then the wrapper blocks, and the agent's own handler runs only after the release. An unheld hook passes straight through (synchronously on js-chat-host), so wiring it changes no timing.

  • The "inside" probe is isInside('onAgentSuspended', sid): a caller is blocked in the hook right now. isHeld alone only says a hold is in effect (it stays true after an evicted caller was let go).
  • Where the window is. Every runtime fires onAgentSuspended AFTER the suspension is persisted and BEFORE the run returns. While it is held the session reads paused (persisted, and the DO's /status), the run has not ended, and on cf-do the DO still executes (hostStatus().isExecuting). Observed (2026-09-27, both hosts, host tier): the chat stream is still open and has carried tool-input-available but not data-run-paused, and the snapshot route reports status: 'active'. So a host-tier responder at 'paused-end' (or 'input', which waits for the exchange to end) cannot answer while the hook is held. Neither can the client tier's responder, at either at: its handler runs as soon as the tool input arrives, but the product Chat's addToolOutput only POSTs when the Chat is not 'streaming' / 'submitted', so the answer rides the request's tail check after the stream ends, i.e. after the release. A scenario that submits INSIDE this window needs a submit outside the Chat's own request loop (a direct submit-tool-result action POST), and must assert isInside('onAgentSuspended', sid) is still true once that submit returned; otherwise a submit that landed after the release passes vacuously.
  • Backends. cf-do and js-chat-host, host and client tiers. Not the executor tier: the executor driver has no timing controls or cell contract to report a leaked hold, and on Temporal the hook runs inside the commitSuspendedStep activity, where a long hold hits hazard H1 below. A scenario that needs it is a declared gap there.
  • Cell contract. As for host-hook holds (see "Release discipline"): a hold in effect at endCell is a HarnessError naming agent hook 'onAgentSuspended' for session '<sid>' (released in endCell's finally: the bridge's on cf-do, js-chat-host's own on JS); a disposed driver releases and names the holds it placed; a caller whose connection closes (an eviction) is let go at once.
ts
// needs: [..., 'hold-agent-hook']
const hooks = driverAgentHooks(driver);
await hooks.hold('onAgentSuspended', sid); // BEFORE start: the bracket opens here
try {
  const run = await driver.start(agent, 'ask', sid);
  const inside = await settle(
    ids.inside,
    hooks.waitUntilEntered('onAgentSuspended', sid, HELD_BOUND_MS),
    HELD_BOUND_MS
  );
  if (inside.settled) {
    await check(ids.windowOpen, async () => {
      expect(await hooks.isInside('onAgentSuspended', sid)).toBe(true);
      // cf-do only: the DO still executes while the session reads paused.
      expect(await driverOp(driver, 'hostStatus')(sid)).toEqual({
        isExecuting: true,
        sessionStatus: 'paused',
      });
    });
    // ...act inside the window (e.g. submit the client tool's result)...
  }
  await hooks.release('onAgentSuspended', sid); // the bracket closes here
  await settle(ids.outcome, run.outcome(), OUTCOME_BOUND_MS);
} finally {
  await hooks.release('onAgentSuspended', sid);
}

Tool gates. hold(sid, gateId), release, isHeld and waitUntilEntered(sid, gateId, boundMs). The gate is entered by the bundled conf-tool-gate agent's gated_step tool (agents/tool-gate.ts), whose scripted { gateId } argument names the gate. The cf-do worker hands the agent a toolGate port (agents/ports.ts) that calls the bridge's tool.gate; the executor driver hands it an in-process port (harness/in-process-ports.ts) behind the same driver.toolGates surface, and the observe-tool-witness capability (driverToolWitnesses(driver)) reads which tool calls actually ran. Built without a port (on Node, or on a host that wires none), the tool refuses to run, so it can never pass an ungated gate by accident. A new gated agent takes its ports from its registry factory the same way.

Run-start commit holds (hold-run-start-commit, CP-96; hold-entry-commit, 16A H4).driverRunStartHolds(driver): hold(sid, entryId, at), release, isHeld, isInside, waitUntilEntered(sid, entryId, boundMs), which resolves with the entering commit: its runId, at, and what its StartRun expects (expectCurrentRunId, expectCurrentRunStatus), read verbatim, and waitUntilSettled(sid, entryId, boundMs) (below). The hold blocks only the calls that carry startRun, the guarded run-start commit of an execute() / resume() / retry() entry. The capture rules live in one registry (harness/run-start-holds.ts) behind two carriers: on the JS and Temporal lanes a wrapper of appendMessages on the env's OWN store object; on the CF Workflows lane a wrapper of the workflow body's D1StateStore in the worker (harness/cfw/worker/run-start-hold-store.ts), which asks the bridge.

  • Two capability names, one member. hold-run-start-commit (JS lanes) also PROMISES that a concurrent entry reaches its commit (the CP-96 stale-entry window). hold-entry-commit (JS, Temporal and CF Workflows) makes no such promise: on Temporal and CF Workflows a concurrent entry of a session whose run is live is refused BEFORE its commit, or waits for the previous instance first. Declare the one your scenario relies on; a scenario that holds only an entry's OWN commit needs hold-entry-commit.

  • waitUntilSettled. After release, it resolves with the store's own answer to the held commit, a product answer: { committed: true }, or { committed: false, error: { name, code?, conflictCause?, rejectCause? } } read verbatim off the thrown error (never its message). A re-attempt of the same run (an activity or step.do retry) that arrives while the hold is in effect goes through the same hold, and EVERY attempt's answer is recorded: waitUntilSettled waits until each attempt that went through the hold and was not abandoned (its caller's connection closed before its commit ran, so it never reports) has answered, and resolves with their common answer. A HarnessError when the attempts DISAGREE (for example a retried attempt committed while the original lost: no single answer is the run's), when a store threw something that is not an Error (named in the message; it has no class identity to report), when no hold was armed, it was never entered, it is not released yet, or the answers do not all come within boundMs. A re-attempt that arrives after the release passes straight through and is not counted. An after-commit hold is entered only by a commit that landed, so it always settles { committed: true }.

  • Temporal. The worker's initializeAgentState / startRunCommit activities commit through the env's store object. A held commit stalls a durable activity: on the time-skipping test server it can time out and be retried (hazard H1 below). The retry carries the same run id, so it goes through the same hold and settles the same way; keep the hold short (until a competitor is proven live, not across a long wait) and run the cell on BOTH HELIX_TEMPORAL_BACKEND servers before ledgering anything.

  • CF Workflows: zero cost unarmed. Arming a hold first arms it on the bridge, then pushes the set of armed sessions to the worker (RUNSTART_ARMED_PATH, its echo compared by content); only an armed session's commit asks the bridge (runstart.enter before the commit, runstart.settled after it), and the set is cleared at cell end (and at the next cell's start if that failed, as for the cf-do switch). A failed runstart.settled report never replaces the store's own answer: the wrapper still returns or throws exactly what the store did, and the failure is a recorded harness failure that fails the cell's endCell. A session no hold was armed for makes no bridge call (owner ruling R8; the cfw self-test checks it at the bridge).

  • Keyed by session and entry. hold(sid, entryId, at) arms a hold that captures the NEXT run-start commit of sid that no earlier-armed hold of sid captured (arm order), and binds to that commit's runId. A re-attempt of the same entry (the same runId, e.g. a run-start retry) goes through the same hold; another entry's commit never does. A commit no armed hold captures passes straight through, synchronously. Arm each hold BEFORE starting the entry it is for.

  • at. 'before-commit': the call blocks before the store runs it (the entry planned; nothing written). 'after-commit': the store commits, then the call blocks before it returns to the runtime (the run is current in the store, the runtime's post-commit work, such as the entry-state write, has not started). A commit that throws is not held and never counts as entered.

  • Who has hold-run-start-commit. Only the JS lanes (EXECUTOR_JS_CAPABILITIES, makeJsExecutorDriver in the conformance-js entrypoint): runtime-js commits every run-start through the store it was built with, in this process. On Temporal and DBOS, while a winner's workflow is open, an owner check refuses a concurrent entry before it reaches its commit (Temporal: the executor's assertNoLiveOwner; DBOS: the entry step's assertDbosOwnerless, or the executor's assertNoOpenOwner once the session reads active); the real-engine cp96-entry-interleave probes assert bReachedCommit === false. Secondary: their activities and steps commit through the same store object, but a held commit there stalls a durable activity or step (on Temporal's test server it is retried under time skipping, hazard H1). The CF Workflows lane commits inside the workflow body in workerd. A scenario that needs the CP-96 promise is a declared gap on those lanes (drivers/capabilities.ts); one that needs only hold-entry-commit runs on Temporal and CF Workflows too (dbos-postgres has neither).

  • Prove the window, or crash. Wrap waitUntilEntered in a named settle() whose bound is LONGER than the hold's own: a hold that is never entered then rejects with the capability's HarnessError, which settle() turns into a cell crash. Use the entered record to prove which plan the entry made, and read the store (observe()) for anything else the window needs. A window that did not open must crash the cell, never pass it.

  • Cell contract. A hold still in effect at endCell is a HarnessError naming run-start hold '<entryId>' (<at>) for session '<sid>', released there; endCell also forgets the cell's holds and rejects pending waits. Dispose releases every hold silently and restores the store's own appendMessages.

control.stale-entry-loses-after-winner (scenarios/stale-entry.ts) is the worked example. It holds entry A after-commit and entry B before-commit, then proves B's commit expects A's fresh run. It releases A, proves in the store that A is terminal and that B is still held, and only then releases B:

ts
// needs: [..., 'hold-run-start-commit', 'observe-persisted', 'observe-run-records']
const holds = driverRunStartHolds(driver);
await holds.hold(sid, 'a', 'after-commit');
await holds.hold(sid, 'b', 'before-commit');
try {
  const runA = await entry();
  const a = await settle(
    ids.aCommitHeld,
    holds.waitUntilEntered(sid, 'a', HOLD_ENTER_BOUND_MS), // shorter than the settle
    HOLD_SETTLE_BOUND_MS
  );
  const runB = await entry();
  const b = await settle(
    ids.bCommitHeld,
    holds.waitUntilEntered(sid, 'b', HOLD_ENTER_BOUND_MS),
    HOLD_SETTLE_BOUND_MS
  );
  // ...B's commit must expect A's run, else throw a HarnessError...
  await holds.release(sid, 'a');
  const aOut = await settle(ids.aSettles, runA.outcome(), RUN_OUTCOME_BOUND_MS);
  // ...A terminal in the store and B still inside its hold, else a HarnessError...
  await holds.release(sid, 'b');
  const bOut = await settle(ids.bSettles, runB.outcome(), RUN_OUTCOME_BOUND_MS);
} finally {
  await holds.release(sid, 'a');
  await holds.release(sid, 'b');
}

A typed loser is judged by RunOutcome.thrownError (executor tier): the thrown error's name and its code / conflictCause / rejectCause fields, read verbatim off the object. thrown is the classified ErrorDetail, which keeps no class or cause. Never match on the message. An entry whose loss the CALLER never sees (a parent's companion resume, whose run-start commit runs inside the parent's tool) is judged the same way through waitUntilSettled's error.

Completion support (refuse-completion-model, switch-completion-support, 16A H1 / H2). Every lane that declares refuse-completion-model applies the conformance completion-support rule (conformance/agents/completion-support.ts) in its LLM adapter: the in-process lanes through the lane ScriptedModel's resolveCapabilities option (createConformanceModel, harness/completion-support.ts), the cf-do DO and the CF Workflows body through their ProxyModel. An outputSchema agent on CONF_UNFINISHABLE_MODEL ('conf/unfinishable') is then refused by the PRODUCT's own completion check (framework_completion_strategy_unavailable, message prefix Model 'conf/unfinishable' (capability profile 'conf-refuse') supports neither forced tool choice) at every entry; every other model id resolves to exactly LEGACY_RESOLUTION, so no other agent changes (__tests__/completion-support.test.ts pins it). dbos-postgres does not declare it: DBOS asks no adapter until completion strategies MR 3, so a scenario that needs a refusal is a declared gap there. Assert a refusal by its code and a message PREFIX, never the full text (the hosts sanitize and may truncate it).

switch-completion-support (cf-do and js-chat-host, host and client tiers): driverCompletionSupport(driver).set(CONF_SWITCHABLE_MODEL, 'refused' | 'default') switches 'conf/switch' for the rest of the cell; the switch is reset at every cell end. On js-chat-host the in-process model reads it at once. On cf-do set pushes the table to the worker (an entry route, module state the DOs share) and resolves once the worker holds it; the harness DO subclass's ProxyModel (never DOAB) reads it synchronously, and an eviction re-pushes a switched table. A cell that never switches costs no request: a per-request read from the bridge was tried first and changed the timing of the cf-do client-tool races (CP-53's control). The worker's table is reset at the end of a cell that switched it, and again at the next cell's START if that reset failed (harness/cell-boundary-sync.ts, CellContract.beginCell); a reset failure never masks the cell's own endCell error (both are reported). Switch inside a window you proved open (for example a held onAgentSuspended), never by timing. Switching completion support across an evict() is unsupported until a self-test covers the re-push: a fresh isolate starts at the default table, and a request the DO makes itself before the re-push lands (an alarm) reads the default.

Seeding a companion (seed-companion, 16A H3).driverOp(driver, 'seedInterruptedCompanion')(parentSessionId, { parent, agent, name, task }) writes, store-direct, an INTERRUPTED persistent companion name of the parent: the child session, its task row under one seed run that ends interrupted, the child interrupted, and the parent's interrupted ref (completionDelivered: true) — the shape runtime-js itself leaves for an interrupted companion (CP-68). Both agents must come back from prepare() (Temporal resolves the child by name) and the parent's persistentAgents must list the child; the parent must exist with no live run. Every precondition and store failure is a HarnessError. It returns { childSessionId, seedRunId }, and the cell's contract covers the child session.

Integrity (owner ruling R9): seeding is setup only. It must never produce the state a cell asserts. It writes product state directly, as seed does, so a cell may use it only to reach a starting point the product path cannot reach on that lane, and must then judge only what the PRODUCT does from there. CP-73 is the counter-example: its cell asserts that the parent's ref reads interrupted after a real interrupt, and Temporal never writes that ref. Seeding the ref would hand the cell the very value it asserts and pass it vacuously, which is why CP-73 needs a real interrupt of a session the scenario did not start, not a seed. Give a seeded cell a positive control where the product's acceptance of the seed is in doubt (the CP-103 refusal cell's control-resumed leg resumes a seeded companion on a finishable model, so a seed artifact cannot pass as the refusal).

Seeding a suspended DO run (cf-do, through do-request; CP-115).POST /__conformance/seed-suspended-run with { status } (suspended_awaiting_children or suspended_step_partial) on a session's DO starts a new run over its settled turn through the DO store's own guarded run-start commit (no rows), sets that run's status and saves the session paused (SEED_SUSPENDED_RUN_PATH, harness/cf-do/store-read-protocol.ts). It answers { runId, previousRunId }, or 409 when the session has no run or its current run is live. The DO never reaches either status by itself (a remote sub-agent cannot suspend on a client tool), so this is the only way a cf-do cell gets there. R9 applies: the do.start-over-*-refused cells assert only the DO's answer to an entry over the seed, and their .marked-send-refused positive control shows the DO reads the seed as a live suspended run.

Evict. The next request, wake(), or nothing at all wakes the DO: drivers re-open nothing implicitly. A body the eviction severs is the eviction's (host: none / stream:severed-by-evict; client: the Chat's own error), never a harness transport failure.

Wake. wake(sid, { via }) returns what the product answered (WakeResult); assert on it under your own ids. A transport or wiring failure is a HarnessError.

  • via: 'request' (the default): GET the session's chat snapshot route (the page load; the driver names the route from the agent it ran the session with, so waking a session this driver never ran is a HarnessError). PartyServer runs the DO's onStart wake recovery before serving it. Returns { via: 'request', route, status }.
  • via: 'alarm': POST /__conformance/fire-alarms/:sid, which calls the DO's REAL alarm handler through DO RPC: PartyServer's alarm(), which runs onStart on a fresh instance and then DOAB's onAlarm(). Nothing is reimplemented. It is RPC, not a DO fetch, because a fetch would run onStart before any route handler and turn every alarm wake into a request wake. The alarm is fired only if one is armed; armedAlarm is read before anything wakes the DO. It returns one of two shapes:
    • { via: 'alarm', fired: true, armedAlarm, instance, handlerError? };
    • { via: 'alarm', fired: false, armedAlarm: null } when no alarm was armed, so the handler is not run, as in production.

Which recovery an alarm wake exercises (instance). After an eviction the fire meets a FRESH instance (instance: 'fresh'): PartyServer's alarm() runs onStart first, which runs recoverStrandedExecution('wake'), the same hook-less recovery a request wake runs. onAlarm's 'watchdog' recovery then finds nothing left to do. So after an eviction via: 'alarm' and via: 'request' exercise the same recovery; the alarm wake differs only in how the instance is started. The 'watchdog' path is reached only by firing on a WARM instance (instance: 'warm', no eviction). A scenario that claims alarm-path coverage must say which of the two it exercised (for MA-09's wake hooks, an evicted DO covers the 'wake' recovery only).

Since Miniflare 5, an evicted DO's durable alarm also fires on its own at its deadline, about 30s after the run started, just as in production. (Under Miniflare 3 it never fired after dispose + recreate.) So a wake scenario must wake the DO well before that deadline, or the alarm, not the scenario's wake, runs the recovery. Waking early and seeing the DO asleep first are not enough on their own: a wake scenario must also prove its own wake ran the recovery. For example, materialize.wake-hooks' .recovered-by-the-wake requires the recovered model call to start after the wake request and before any alarm could be due (under 30s after the run started), and the timing-controls request-wake self-test checks the same against the alarm it read while the DO was awake. Without that, a broken request wake can pass through the alarm. (The fixed 30s bound can fail loudly, never silently, under extreme load; see materialize.ts.) The on-demand via: 'alarm' fire differs from production in three ways: it fires now rather than at its deadline, a handler throw is reported (handlerError) instead of retried with backoff, and the handler gets no alarmInfo. See FU-CONF-DO-MINIFLARE-ALARM-ACROSS-EVICT in docs/dev/follow-ups.md. Name the wake your scenario uses in its title or comments.

Mid-text model holds (holdAfterDeltas, timing control B). A ScriptedModel script entry, not a driver capability, so it needs nothing in needs and works on every lane that runs the ScriptedModel: the executor lanes, js-chat-host, and cf-do (whose model runs in the Node bridge).

ts
import { holdAfterDeltas, scriptedTextDeltas } from '@helix-agents/core/testing/mocks';

const TEXT = 'ALPHA BRAVO CHARLIE DELTA';
await model.scriptFor(
  sid,
  holdAfterDeltas(2, gate, { type: 'text', content: TEXT, shouldStop: true })
);
const run = await driver.start(agent, 'hi', sid);
try {
  // The inside probe: paused with exactly 2 deltas out.
  const rec = await model.whenHeld(gate, HELD_BOUND_MS); // rec.heldAfterDeltas === 2
  const partial = scriptedTextDeltas(TEXT).slice(0, 2).join(''); // 'ALPHA BRAVO'
  // ...act mid-text: reload, attach with a cursor, read the snapshot...
} finally {
  await model.release(gate);
}
await settle(ids.outcome, run.outcome(), OUTCOME_BOUND_MS);
  • What it does. holdAfterDeltas(n, gateId, then) streams then (a text response, or a tool_calls response with content) exactly as it would unscripted — MockLLMAdapter's word chunks, which scriptedTextDeltas(text) returns — and pauses after EXACTLY nonTextDelta calls. Everything after them (the remaining deltas, any tool call, the step end) and the result wait for release(gateId), then stream in their original order. Deterministic: the same n deltas every time. The builder throws a ScriptedModelError for a pause that could never happen (n not a positive integer, more than the text has, no text, an empty gate id); so do scriptFor / script / scriptForAgentType for such a hand-built literal.

  • The "inside" probe is model.whenHeld(gateId, boundMs) (the same gates as hold). It resolves only once the call is paused AND its n deltas are out: each forwarded onTextDelta's own promise (the runtime's stream write) settled, and on cf-do the worker's proxy acked the bridge's delivery barrier (a barrier line it acks with model.barrier after every earlier line's callback ran in the DO). The record then carries heldAfterDeltas: n, and model.outstandingHoldsAfterDeltas() lists the call. "Out" means in the host's runtime, not yet at the caller: wait for the caller's own view (a settle) before judging it. Observed (2026-09-27): on cf-do and js-chat-host the snapshot route then returns the partial text with status: 'active'.

  • Abort and reset. An abort while paused (a stop, an eviction's closed connection on cf-do) answers framework_cancelled, like hold; the rest of the stream is never delivered. model.reset() (every cell start and end) aborts a paused call the same way, so a backend is never wedged.

  • Cell contract. A pause still in effect at cell end is a leaked hold: runCell makes it a cell-contract failure on EVERY lane (the observation is discarded; the message names ScriptedModel holdAfterDeltas gate '<gate>' (session '<sid>', <n> delta(s) out)), then its reset aborts the call. The same check covers a plain hold left blocked (ScriptedModel hold gate '<gate>' (session '<sid>') still held at cell end; ScriptedModel.outstandingHolds() is what it reads). The invariant is: on EVERY path, each model hold is released or aborted by cell end. Today's scenarios meet it in different ways:

    • stop.ts, transcript-heal.ts and stream.ts release in a finally;
    • lifecycle.ts (the pause-for-resume setup) has no finally: it relies on the stop's abort, and releases only when the abort does not arrive within STOP_ABORT_BOUND_MS. On its precondition-failure path (the call was not held within awaitCallHeld's bound) it returns WITHOUT releasing, so a call that is held just after that bound is still blocked at cell end;
    • materialize.ts (materialize.wake-hooks) never releases: the eviction aborts the held call (the bridge sees the connection close).

    A path that misses it (such as lifecycle.ts's late hold above, or an eviction abort that never reaches the model) is now a cell-contract crash naming the hold, not only the precondition failure it recorded before. One exception: a hold in the session of a DESCENDANT execution (a sub-agent or companion child workflow under this cell's own roots) that the cell-end REAP terminated, when no run this cell tracks has that session, is the harness's own cut — on Temporal a workflow terminate never aborts the activity holding the model call — so it is recorded in the cell's reapedHolds (informational) instead, and the model reset aborts it. A hold in a ROOT or tracked run's session is never exempt, even when the reap terminated that run. See FU-CONF-HOLD-CHECK-HIDES-UNDELIVERED-ABORT for triaging the crash. On cf-do the backend's endCell also reports the in-flight model call.

  • Temporal. Hazard H1 below applies as to any scripted-model hold: keep the pause short there.

Release discipline. Release every hold in a finally (see "Held work must always release"). The harness backs that up in two places, and neither ever hangs a cell:

  • a hold still in effect at endCell makes it throw a HarnessError that names the hold (for example host hook 'onRunSettled' for session 'x' or agent hook 'onAgentSuspended' for session 'x'); the cell is a crash and the hold is released in endCell's finally (a model hold / holdAfterDeltas left blocked is caught by runCell's own cell-end check, above);
  • a driver disposed with a hold it placed still in effect releases it FIRST (so a run it blocks can drain) and then throws a HarnessError naming it. Holds endCell already released are not reported again.

A held caller whose connection closes (an eviction while the hook, tool or model result waits) is let go at once; the hold itself stays in effect until it is released.

Only the leaking cell fails (cf-do: the leak fence). Tearing a leak down happens AFTER the cell's endCell closed its sessions: endCell releases the hold, and runCell's model reset aborts a call left in flight. The session's run then reacts (its hook replies, the aborted call's onAgentFail and onRunSettled, a cancel or barrier ack), all of it traffic for a closed session. The bridge FENCES that aftermath for every session whose own endCell reported a leak of it (a model call in flight, or a hold in effect): it is kept in bridge.late with fenced: true, never a failure of the next cell, which would otherwise crash for a cause it did not have. A new MODEL CALL from a fenced session is never fenced (it would consume the next cell's scripts), and late traffic from a session whose cell ended cleanly still fails the next endCell. Only the LEAKED sessions are fenced: a bystander session of the same failing cell is not. The fence is epoch-scoped: it covers the rest of the teardown and the ONE cell after the leak (in the suite, the crash's backend rebuild, whose dispose runs the bridge's final health check) and is pruned when that cell ends. Aftermath that outlives a further cell fails loudly.

Examples ​

These are the shapes the live self-tests in harness/cf-do/__tests__/timing-controls.integ.test.ts drive through both drivers (agent-hook holds and holdAfterDeltas: harness/cf-do/__tests__/hold-controls.integ.test.ts, on cf-do and js-chat-host). In a scenario, wrap every wait in a named settle() and record results with check(). The real scenarios built on them are the worked examples in "The host and client scenarios" below: the admission race is scenarios/send-during-entry-admission.ts, evict and wake is materialize.wake-hooks (scenarios/materialize.ts), and a reload mid-message is the replay leg of stream.ui-chunk-schema-client (scenarios/stream.ts).

Slow onRunSettled. Hold the settle hook, run, and observe that the DO no longer executes while the hook is held (the settle path released it first):

ts
const hooks = driverHostHooks(driver);
await hooks.hold('onRunSettled', sid);
try {
  const run = await driver.start(agent, 'hi', sid);
  await hooks.waitUntilEntered('onRunSettled', sid, HELD_BOUND_MS);
  // ...observe here: the run has ended, the DO is NOT executing, the hook is held...
  await settle(ids.outcome, run.outcome(), OUTCOME_BOUND_MS);
  await hooks.release('onRunSettled', sid);
} finally {
  await hooks.release('onRunSettled', sid);
}

Two entries in admission (CP-02). Send A is held in admission; send B reads the session as completed and is refused at the DO's producer claim, which A holds. Assert the window was OPEN before judging B: A's hold was entered (.first-entry-held), and a hold that never opened is a named precondition failure, never a silent "the race did not happen" pass. Fire A without awaiting it: its request answers only once A is released.

ts
// A condensed scenarios/send-during-entry-admission.ts: one id per wait or check.
const entries = driverEntryHooks(driver);
const send = driverOp(driver, 'send');
await entries.hold(sid, 'TURN-2A'); // BEFORE A's send: the bracket opens here
try {
  const aRun = send(agent, sid, 'TURN-2A'); // not awaited
  aRun.catch(() => {});
  const aHeld = await settle(
    ids.firstHeld,
    entries.waitUntilEntered(sid, 'TURN-2A', RUN_OUTCOME_BOUND_MS * 2),
    RUN_OUTCOME_BOUND_MS
  );
  if (!aHeld.settled) {
    await failPreconditions(check, later, ids.firstHeld);
    return;
  }
  // B, while A is held: a typed 409 `state_already_running`, or an answered turn.
  const bOut = await settle(
    ids.sendSettles,
    send(agent, sid, 'TURN-2B').then((r) => r.outcome()),
    RUN_OUTCOME_BOUND_MS
  );
  await check(ids.notDropped, async () => {
    /* ...a typed retryable 409, or TURN-2B and its reply persisted... */
  });
  await entries.release(sid, 'TURN-2A'); // the bracket closes here
  // ...then A's own turn completes (`.first-entry-completes`).
} finally {
  await entries.release(sid, 'TURN-2A');
}

The proof that B met A's admission is the bracket: the hold is placed before A's send and released only after B's answer was judged, and the producer claim is held across the whole admission (runDrive).

Evict and wake. Hold the model, evict mid-call, confirm the DO sleeps, then wake it through the product. Scenarios that want the product's alarm-driven wake use via: 'alarm':

ts
await model.scriptFor(
  sid,
  { kind: 'hold', gateId: gate, then: { type: 'text', content: 'lost', shouldStop: true } },
  { type: 'text', content: 'AFTER-WAKE', shouldStop: true } // the recovered run's call
);
const run = await driver.start(agent, 'hi', sid);
await model.whenHeld(gate, HELD_BOUND_MS);
await driverOp(driver, 'evict')(); // the held call is aborted (connection closed)
await settle(ids.severed, run.outcome(), OUTCOME_BOUND_MS);
const woke = await driverOp(driver, 'wake')(sid, { via: 'alarm' });
await check(ids.alarmFired, () => {
  if (woke.via !== 'alarm' || !woke.fired)
    throw new Error(`no alarm was armed: ${JSON.stringify(woke)}`);
  // After an eviction: 'fresh' — onStart's 'wake' recovery did the work.
});
// ...settle until the recovered run completes, then observe its hooks.

Reload mid-message, tool-gated (client tier). Stream some text, hold the run in a tool gate so it stays 'active', and reload. The new hook instance is built from the snapshot and resumes:

ts
await model.scriptFor(
  sid,
  {
    type: 'tool_calls',
    content: 'PART-ONE ',
    toolCalls: [{ id: 'c1', name: GATED_TOOL, arguments: { gateId: 'g' } }],
  },
  { type: 'text', content: 'PART-TWO', shouldStop: true }
);
const gates = driverToolGates(driver);
await gates.hold(sid, 'g');
try {
  const run = await driver.start(toolGateAgentPrepared, 'go', sid);
  await gates.waitUntilEntered(sid, 'g', HELD_BOUND_MS);
  await driverOp(driver, 'reload')(sid); // the old instance reports its own end
  await gates.release(sid, 'g');
} finally {
  await gates.release(sid, 'g');
}
// observe(sid).client is the NEW instance: a snapshot GET, then the resume GET.

This is NOT a mid-text reload: the model's own hold blocks BEFORE the call streams anything, so the reload lands mid-message (the step's text is out and the run is live in the tool). For a true mid-text reload, pause the model with holdAfterDeltas instead (FU-CONF-TRUE-MID-TEXT-HOLD: the primitive exists; the scenario is S5's). Until DI-20's fix (!268) the client also errored on finish-step before the reload. Now the first instance ends client:unmounted, and the new one resumes.

The host and client scenarios (worked examples) ​

One scenario per caller channel. The caller's view is the stream channel at the host tier and the client channel at the client tier, and needs are per scenario, not per tier. A scenario that read both channels would need both observe-stream and observe-client, so it would be a declared gap on every lane. So a case that judges what the caller saw is a factory with a caller parameter that builds one scenario per tier: submit.no-race (host) and submit.no-race-client (client), and so on. Everything tier-neutral (the outcome, persisted, llm, hooks, the timing controls) is shared. A tier-neutral scenario, such as smoke.hello-completes or materialize.wake-hooks, is ONE scenario over several tiers.

The client-tool round (scenarios/submit.ts). submit.no-race / submit.no-race-client start the client-tool round with the responder answering CONTROL_DELAY_MS after the paused end, and judge it. The caller-side ids are split so that one cause gets one id. .submitted means the caller sent the auto-send at all. It exists for the client tier: until DI-20's fix, the product Chat erred on the step events before it answered, which is not a submit failure. Its dependents fail as its preconditions. .submit-accepted and .continuation-reaches-caller are the caller's view; .run-completes, .model-sees-result-once and .not-stranded-paused are the server's. On cf-do the submit wakes the idle DO and the handler's own /resume then 409s, so the caller gets only data-resume-rejected (CP-53).

They began as race A's control. Race A (submit.race-a*, CP-52) held the DO's host onComplete so the DO was still executing the paused run when the submit landed. WA MR1 retired it (Task 20): the settle hook now runs after the execution release, and nothing between the paused frame and the release awaits, so the window no longer exists in production.

Attribute per scenario, by mechanism, not by matching ids across scenarios. Race A and its control failed the same two caller-side ids for different reasons: in the control the DO was idle and the handler's own /resume raced the woken run (CP-53); in race A the DO was still executing the held run and the run was stranded paused (CP-52), whose record explained all of race A's failing ids. Once CP-52 was fixed, race A's caller-side ids failed under CP-53 instead, re-attributed from the observed run, never pre-assigned. Build a control whenever the window you hold could be confused with a defect that fires without it.

A defect that ends the caller first masks everything after it. DI-20 is the worked example.

Before DI-20's fix, the client tier's product Chat rejected the finish-step chunk before anything else the client cells of smoke.hello-completes, submit.race-a-client and submit.no-race-client judge happens. No tool handler ran and nothing was submitted, so every id after .submitted failed as its precondition. Those ids were ledgered to DI-20. That is step 2 of §4's decision order: DI-20's mechanism fully explained each id as observed, and no other cause could show while nothing reached the host.

When the fix landed (!268), the masked cells were re-run and each id was re-attributed from the new observation (FU-CONF-DI20-MASKED-CLIENT-CELLS):

  • the smoke cells and submit.no-race-client|js-chat-host|client passed. They are cited in DI-20's record as corroboration, NOT in its provenBy: they assert nothing about step frames, so they would keep passing if the hosts stopped emitting step events. A provenBy cell must observe the thing the fix changed (here, the stream.ui-chunk-schema cells, which require the step frames to be present);
  • submit.race-a-client|cf-do|client (since retired) failed race A's host shape and moved to CP-52;
  • submit.no-race-client|cf-do|client failed the control's host shape and moved to CP-53.

Never move the fragments to the finding you expect in advance. If an id could have a second cause even while masked, it is not fully explained: ask the owner.

The settle hook releases (lifecycle.settle-hook-releases). The "Slow onRunSettled" shape above, spanning the next turn: with turn 1's settle hook held, the DO reports it is not executing and a turn-2 send is answered. It is WA MR1's contract change and its regression guard (it would fail if the hook moved back before the release).

The admission race (lifecycle.send-during-entry-admission, CP-02). The "Two entries in admission" shape above: B's handler read completed, took the fresh-turn path and met AlreadyRunning at the producer claim, the fallback CP-02's fix changed.

Evict and wake (materialize.wake-hooks, MA-09). The model call is held, the backend evicted, the severed request settled, and then the DO is woken with wake(sid, { via: 'request' }) (the snapshot GET). Between evict() and wake() NOTHING may touch the host: observe() (the persisted store reads) and hostStatus() are DO requests and would wake the DO themselves, running the recovery before the scenario's own wake. The severed request's outcome, which is the caller's view, is the only thing awaited in between. After an eviction the wake meets a fresh instance, so the scenario claims only onStart's 'wake' recovery, not the warm 'watchdog' path. After the woken run completes, a short quiet period comes before the hooks are read, so a duplicate late hook is still counted.

Reload mid-message (stream.ui-chunk-schema-client, replay leg). The model call is held while the run is live. reload(sid) builds the new instance from the snapshot, which resumes because the session is 'active'. The call is then released, and a named poll waits until the new instance is quiescent and the run completed before .client-accepts-replay reads it.

3. The integrity rules ​

Each of these is enforced mechanically — violating it doesn't just produce a worse scenario, it produces a broken or unsafe one.

  • Named assertions only. Every failure is attributed to a declared id. AssertionRecorder.check/.settle throw a HarnessError on an undeclared id (assertions.ts's requireDeclared).

  • Every bounded wait is a named ctx.settle(id, promise, boundMs) — never raw setTimeout, Promise.race, or vi.waitFor. A settle() timeout is a real, ledgerable assertion failure under id AND marks the cell unsettled (suite.ts's BackendLane disposes and rebuilds the backend before the next cell — the run may still be executing). Dependent checks unwrap the result with settled(r, id) (assertions.ts), which fails the calling check with precondition <id> failed rather than crashing when the settle it depends on never resolved. An optional 4th argument, describeTimeout: () => string, is called only when the bound fires; its text is appended to the timeout's failure message, so the failure can name what was stuck (session.create-observed-whole names the session and the phase of the attempt that blew one of its per-stage guards).

  • ctx.recordTiming(timing) is informational. A CellTiming is { label, ms: Record<name, number>, unfinished? }: measured milliseconds by name, plus the part still running when the timing is of unfinished work. runCell writes every recorded timing to the cell's result line as timings, even when the cell crashed or hit its outer timeout. It never changes a verdict and is not an assertion. Record what a CI artifact needs to explain a slow or failed cell (session.create-observed-whole records each attempt's phases; stream.suspension-marker its window's slack).

  • Every check()/settle() call is awaited. lint:conformance's type-aware no-floating-promises rule and AssertionRecorder.outstanding (runtime check at cell end) both catch an un-awaited one — an un-awaited call's late failure would otherwise land after the result was already taken (a silent pass).

  • check() is an allowlist, not a denylist. Only the exact error shape a real expect() failure throws (name === 'AssertionError') gets recorded under the checked id. Anything else — a HarnessError, a plain Error from a store/driver bug, a bug in the check body itself — rethrows and crashes the cell (<id>:crash). Never wrap a harness error in a try/catch and turn it into an expect() failure yourself. The one sanctioned exception is ShortReadError via requirePersisted/requireHistoryReadable (assertions.ts) — a short/failed persisted read the scenario is explicitly allowed to recognize as the expected shape of the defect under test (see types.ts's HarnessError doc comment for the fuller ObservationError case).

  • Never synthesize an observation. No default errorDetail, no hooks ?? { completed: 0, ... }, no fallback message list. A missing channel is a HarnessError, not a stand-in value — a stand-in would let a broken observation path masquerade as a real (possibly passing) result.

  • Assert outcomes only: what the model received, what persisted, what the client saw, terminal status, errorDetail. Never internal call order or call counts beyond what's needed to prove the outcome.

  • Scenarios never branch on runtime/backend. A driver that can't express a needed primitive is a declared gap (owner-managed, needs); a product lacking a capability is requires + rejection asserting a typed framework_not_supported — never an if (backend.runtime === 'temporal') inside a scenario body.

  • Runtime imports are restricted, to keep the scenario side portable to a future workerd pool (spec §4.1 rule 6): scenarios/**, cell.ts, timeout.ts, assertions.ts, judge.ts, types.ts, drivers/session-driver.ts, observation/observation.ts and canaries/** import, AT RUNTIME, only @helix-agents/core, vitest, zod, and each other — never node:*. An ../harness/* import must be import type only (erased at compile time, so it loads nothing in either pool). catalogue/** records import only ../define.js — no vitest, no node:* — so findings.ts/ledger.ts stay importable from a plain tsx script (e.g. conformance-docs.ts). Nothing enforces this but code review today; a violation compiles and passes locally, then breaks when that module is later bundled into the workerd pool.

  • Bundled scenario agents are stricter, and their rule is ENFORCED by lint:conformance (no-restricted-imports in eslint.conformance.config.js). The modules under conformance/agents/ are imported by Node scenarios AND bundled into the cf-do worker (workerd), so at runtime they import only @helix-agents/core, zod and sibling agent modules; anything else (e.g. ../types.js) only as import type; never vitest and never a Node builtin (bare or node:). A scenario that runs builds its agent with the registry factory (AGENTS, e.g. smokeAgent()), unmodified — every executor-tier agent is bundled (see "Scenario agents"); the cf-do driver's prepare() fingerprints it against the bundled agent and the executor driver's prepare() fingerprints its rebuild against it, and a drift is a HarnessError. The fingerprint compares function-valued fields (a tool's execute, hooks, a function systemPrompt) by presence only, because their source text differs between the Node and workerd builds. So prepare() also demands provenance (agents/provenance.ts): the agent object and each of its tool objects must have come out of a registry factory. A spread copy or a replaced tool is refused. Mutating a factory-built object in place is not caught, so never do it.

  • Bounded waits must fit under the scenario's cell timeout.scenario-budgets.test.ts (owner-only) asserts timeoutMs ≥ <the longest path's summed settle bounds> + POST_SCENARIO_OVERHEAD_MS + MARGIN_MS for every scenario it lists. Export your scenario's worst-case constant (see stop.ts's MID_TOOL_WORST_CASE_MS) and ask the owner to add a budget-test row whenever you introduce a new long wait or raise timeoutMs. Skip this and a genuinely slow runtime blows the OUTER cell timeout instead of your named settle() — an anonymous <scenario>:crash instead of the ledgerable, attributable failure you actually want.

  • cf-do bridge cells: the observation is valid only if endCell passes. On the Miniflare-backed cf-do backend (packages/e2e/src/harness/cf-do/bridge-server.ts, "THE CELL CONTRACT"), the enforcement is endCell, which runCell runs; no driver calls assertHealthy today (it is an optional earlier, scoped probe, used by the harness self-tests). runCell ends every cell itself when the backend env has a cellContract (BackendEnv.cellContract, carried onto the suite's BackendInstance): after the leftover-run grace and the model checks, before the model reset, it calls cellContract.endCell(<every session id the cell touched through the driver, plus every sub-session an observation reported>). If endCell throws, the observation is discarded: the cell's result becomes a single <scenario>:crash naming the contract failure (the discarded results are listed in its message for diagnosis only) — never a pass and never a ledgerable product result. endCell sweeps every failure: the listed sessions', unscoped ones and late ones (a scoped assertHealthy would see only the first kind). Session ids and model callIds are single-use across cells. Any traffic or failure for a closed session is a HarnessError. Practical consequence: a cf-do scenario must leave no model call in flight and no hold outstanding at its end (release in finally, settle the run) — endCell reports them and the cell crashes. Backends without a cellContract (the Node executor lanes) are unaffected.

  • cf-do persisted reads are several requests, not one snapshot. On cf-do the persisted channel reads the session's DO-SQLite through one HTTP round trip per store call (loadState, getMessageCount, each getMessages page, each checkpoint). Against a LIVE run those reads can tear: the run may commit a step between them, so messages, status, checkpoints and latestCheckpointId can describe different moments. (readPersisted re-reads once on a count mismatch and throws ShortReadError only if it persists, but it cannot make the other fields agree.) Observe a cf-do session only after a named settle() has seen the run finish, or while the run is held (e.g. its model call held with model.hold), never mid-flight. The same holds for the host tier's status probe. Also note that a cf-do read or status probe is a real request: it wakes an evicted DO (its wake recovery runs first).

A cell never leaks running work: cell end, the reap, and dispose ​

A lane reuses ONE backend instance across its cells and rebuilds it only after a crashed, unsettled or left-running cell, so anything a cell leaves running would execute during the NEXT cells. The harness enforces this at three points. Scenario authors don't call any of them, but they need to know what happens to the work their runs leave behind.

  1. Leftover-run grace. Once the scenario returns, runCell waits up to LEFT_RUNNING_GRACE_MS for the cell's own tracked runs to finish. Runs still unfinished after that are leftRunning. This is informational and never a verdict, but it makes the lane rebuild before the next cell.
  2. The cell-end reap (RunLog.reapSince). After the grace and before the model reset, the executor driver asks the backend to end ALL the work the cell's sessions still have running. That includes work no tracked handle covers. On Temporal, a sub-agent or persistent-companion child runs as its own workflow. It can outlive its parent's run: a parent stop never reaches a blocking companion (CP-23), and a non-blocking companion is ABANDONed by design. Such an orphan kept calling the model after its cell and wrote to its torn-down parent stream. The result was unhandled rejections ("Cannot write to stream … in 'failed'/'ended' state") in later cells, and a red test:conformance although every cell passed.
    • The backend hook is BackendEnv.terminateCellWork(sessionIds, reason, { closeGraceMs }). The Temporal helpers implement it (StartedWorkflows.terminateTree in _temporal-client-adapter.ts), and so does the CF Workflows lane, from its instance ledger (see "The CF Workflows lane" below). JS and DBOS don't, so on those lanes the reap is a no-op and nothing changed. The rest of this item describes Temporal.
    • It walks the workflow TREE of every execution the env's client started for the cell's sessions. Each root is pinned by its first-execution run id. Each child is found in its parent's own history (StartChildWorkflowExecutionInitiated → ChildWorkflowExecutionStarted), so the walk never depends on how runtime-temporal names child workflow ids.
    • It deliberately doesn't use visibility (list-by-query). The in-process Java test server answers every ListWorkflowExecutions with "Failed to list workflows", and the Docker server's visibility is eventually consistent.
    • It first snapshots which executions are RUNNING, then gives them REAP_CLOSE_GRACE_MS (500ms) to close ON THEIR OWN, since a root can still be finishing after its handle settled. Only what is still running after that is terminated, top-down, so each history it reads is final. It then re-describes every execution it found.
    • An execution counts as reaped only if its final status is TERMINATED and the reap ended it: either directly, or through the server's close-policy cascade from a parent the reap terminated. One that closed on its own is never counted.
    • "Stopped" is judged from the server's execution status, not from whether a product handle's promise settled:
      • Still RUNNING after termination is a genuine leak: a HarnessError. The cell crashes, the lane rebuilds, and the rebuild's dispose checks again. If the execution is still open then, the lane is poisoned. If the cell had already crashed, the reap's failure is added as a second message under the same crash id; it is never dropped.
      • Closed, but the run's product handle never settled (even after REAP_SETTLE_BOUND_MS) is never a crash. It is reported by cause, one cause per class, and the run is not waited on again at dispose:
        • terminatedUnsettled: the harness terminated an execution of the run's session. This is session-level, so it may mix causes. Often it is harness-caused: Temporal's stream-observed resume handle never observes an external terminate of its own workflow. But a product-side unsettled run also lands here when the harness terminated an unrelated execution in the same session, such as an orphan companion. So it is never reported as product evidence. Per-run attribution would need the product handle's workflow and run id, which AgentExecutionHandle does not expose.
        • closedUnsettled: every execution closed on its own, and the harness terminated none. This is product-side. The harness never attributes it to a specific finding. A finding is burned down by its own scenario cell with a ledgered id, not by a harness warning.
    • Whatever the reap terminated is reported per cell as reaped, beside leftRunning, leftoverScripts and the scenario's own timings (ctx.recordTiming, see §3). reaped, closedUnsettled and terminatedUnsettled go into the results JSONL and a section of the conformance:verify summary, plus a console.warn in the run log. They're informational and never change a verdict.
    • The reap removes a signal, so assert what matters. Before the reap, a product workflow that outlived its cell eventually made test:conformance exit red (by accident). Now it is ended quietly and listed under reaped. If a child or companion outliving a settled parent matters to the product, assert it as a named id from the scenario (e.g. observe() the child's persisted status). Don't rely on the reap's report.
    • Bounds. Each backend call of the reap (a describe, a history read, a terminate) is bounded by REAP_CALL_BOUND_MS (3 s), which is what detects a hung backend. The whole call is bounded by reapCellWorkBoundMs(runs): REAP_CELL_WORK_BASE_MS (5 s) plus REAP_CELL_WORK_PER_RUN_MS (10 ms) per run the cell started, so a cell that ends many sessions (session.create-observed-whole: 48) gets more than one that ends a handful. The close grace is spent inside it. Its worst case (reapWorstCaseMs) counts against the cell's timeoutMs (scenario-budgets.test.ts's post-scenario overhead, budgeted at the largest cell).
    • Cost. On Temporal an execution seen CLOSED is never read again: its status and its final history's child starts are kept, so it costs 2 backend calls whatever pass or later call reaches it. Up to TEMPORAL_REAP_CONCURRENCY (8) trees are walked at once; within one tree the walk stays top-down. The report keeps the roots' order. The first failure stops the whole call.
    • One walk at a time. When the bound fires, the driver aborts the call: it issues no new backend call and settles once its in-flight calls did. A later call on the same env (the rebuild's dispose) waits for it, then walks only what is left. Two calls never walk the same executions at once.
    • Diagnostics. A timed-out or failed reap names only the executions still unchecked or unended, the number of backend calls made and the time spent (and its stage), not every session the instance ran. A backend that reports no progress (CellWorkProgress) is named by the sessions asked about.
  3. Dispose (stopAndDrain, on every rebuild and at lane teardown). It cancels client-tool chains, ends the runs through the backend, then drains them within DRAIN_BOUND_MS. Its worst case (disposeWorstCaseMs) is held under the suite's DISPOSE_TIMEOUT_MS by scenario-budgets.test.ts, for both paths.
    • Without terminateCellWork (JS, DBOS): terminateInFlight or per-handle aborts, then the drain. A run still in flight after the drain fails dispose, which poisons the lane. This is unchanged.
    • With it (Temporal): one terminateCellWork call over every session the instance ran, bounded by DISPOSE_CELL_WORK_BOUND_MS (10 s, a flat cap; each backend call still by REAP_CALL_BOUND_MS). It first waits for a cell-end walk that timed out and is still settling. It replaces terminateInFlight, which it subsumes: every root the env started for those sessions is in the tree. Then comes the drain.
      • Roots whose whole tree an earlier call (e.g. the cell-end reap) already confirmed closed are NOT walked again. Dispose covers only roots not yet confirmed closed, plus every kept unresolved child start (see the late-child paragraph below).
      • A run with NO handle (its execute() / resume() never returned, or a chain between runs) that did not drain fails dispose, exactly as on JS and DBOS.
      • A run WITH a handle that did not drain is judged again from backend truth at that moment (a second bounded call), so a run started after the first pass is covered too. If every execution is CLOSED, the run isn't in flight, only its handle never settled. The harness warns, attributing it like the reap (harness-terminated vs closed on its own), and the lane continues. An execution still RUNNING fails dispose as a genuine leak.

The real-infra proof is src/harness/__tests__/temporal-cell-reap.integ.test.ts. It covers a blocking companion stopped mid-consult and a non-blocking companion whose parent completed, and runs on both HELIX_TEMPORAL_BACKEND servers. It proves the reap ends the companion and that the server reports it TERMINATED. Its no-escape check did NOT go red with the hook disabled: the probe's orphan does not write to a torn-down stream within the window. The proof that the reap removes the original unhandled "Cannot write to stream" rejections is B2-core's history.companion-continue full-file run, before and after.

A late child start is not missed. A parent's final history can hold a StartChildWorkflowExecutionInitiated with no ChildWorkflowExecutionStarted or StartChildWorkflowExecutionFailed after it. For each such start, the reap looks the child up by id and parentExecution for up to 1s. If it isn't found, the start is kept. Every later reap or dispose call, whatever sessions it is about, looks it up again first. Once the child exists, that call walks and terminates it, attributed to its own session. A kept start still unresolved at dispose is reported with a console.warn, never dropped.

On Temporal's server a closed parent's pending start is never processed: processStartChildExecution returns early, even for ABANDON. So a late start can only come from a start task already in flight when the parent closed. That window is too narrow to force, so it has no live test. The unit tests in temporal-client-adapter.test.ts cover it with a fake history. This closed FU-CONF-TEMPORAL-REAP-LATE-CHILD.

Non-vacuity ​

Every assertion that checks "X is well-formed" must ALSO assert X is present and contains what the scenario actually scripted — otherwise it can pass on an empty or trivially-correct X for the wrong reason, and one guard is rarely enough by itself.

transcript.ts's .persisted-paired first asserts the persisted transcript still contains every turn-1 tool call id the scenario scripted (dropped must be empty) BEFORE calling expectPaired — a runtime that silently dropped a call would otherwise pair trivially (nothing left to be unpaired).

.llm-input-has-turn1-calls and .llm-input-paired share one guard and still each need their own id. Both call the same turn2Input() helper, which first asserts turn 2's model input is provably NEW (llm.length grew past turn 1's), so .at(-1) never reads a stale turn-1 input. That growth guard alone is not enough: on Temporal, turn 2's input DOES grow (a real turn-2 call happened) but carries NO history at all (RM-10) — an empty input still pairs trivially, so .llm-input-paired passes VACUOUSLY there too. What makes the cell loud on Temporal is the SIBLING id: .llm-input-has-turn1-calls independently asserts every turn-1-persisted tool-call id reached the model, which an empty input fails. The pattern: put the shared non-vacuity guard in one helper every dependent check calls, and give each thing that could still pass vacuously its own id checking a different necessary condition (see "One id per cause" below).

stop.ts's .transcript-paired follows the same discipline: it asserts the persisted history contains the scripted user message, and that a stopped step's tool calls are either ALL present or ALL absent (never a partial drop), before calling expectPaired.

Size limits: Miniflare's caps are not production's ​

Miniflare's SQLite accepts a value up to 8 MiB (8,388,608 bytes) in both a Durable Object and D1 (owner probe, Miniflare 5.20260925.0-alpha, 2026-10-19). Production documents 2 MB per value and per row. So locally, a value between 2 MB and 8 MiB is stored and fails only in production, and an oversized cell sized in that range passes for the wrong reason. Rules:

  • Size a cell against the product's configured guard, never against Miniflare's cap. That's the store limit the product enforces itself (for example MA-31's message-row limit, about 1.8 MB), or production's documented cap. The program rule (orchestrator, 2026-10-19) is that product size handling always goes through an explicit configured guard under the production cap, never through SQLite's own SQLITE_TOOBIG. Such behaviour is then the same locally and in production.
  • For a pre-fix red that needs production's cap, declare emulate-production-value-cap and arm it for the cell's sessions, once, as one set (driverValueCap(driver).arm([sessionId, childSessionId])). Armed, the cf-do worker's DO SQL path and the CF Workflows lane's D1 binding (both sides) and StreamServer refuse a bound value over 2,000,000 bytes with the error real SQLite raises. The pre-fix product then fails exactly as in production. A product guard sized under the cap is never masked, so post-fix greens stay green. Values only: a row of many sub-cap values isn't refused.
    • On cf-do the arm is per session DO (kept in its SQLite, so it survives an eviction): arm every session whose writes the cell sizes, a companion's own DO included.
    • On CF Workflows the arm is the lane's (D1 and the STREAMS DO are shared), held for one cell's session set at a time and cleared at cell end. An arm for another set while one is held is a HarnessError, never a silent leak into another cell's writes.
  • A value over 8 MiB fails locally with no emulation. Use it only where nothing else carries the same bytes; escaping can multiply a value on its way into a second row (a companion's output is about 2× larger in its parent's notice), which makes over-8-MiB values unusable for MA-37's shape.

One id per cause ​

When a single check could fail for two different underlying defects, split it into two assertion ids so per-assertion ledger attribution stays honest — the judge compares ids, not messages, so two causes sharing one id would let a fix for one silently start "covering" the other. The .llm-input-has-turn1-calls / .llm-input-paired split above is the worked example: on Temporal the first fails under RM-10 while the second is a separate id attributable to RM-29 on the backends where persisted-pairing is the actual cause, without RM-10 silently satisfying it.

Pairing is adjacency-aware — reuse, never reimplement ​

unpairedToolCalls/expectPaired (transcript.ts) check more than "every tool_use has a tool_result": a result must sit in the run of role: 'tool' messages immediately following the assistant message that issued the call (the misplaced bucket) — a provider rejects a non-adjacent result even if it exists later in the transcript. stop.ts imports and reuses expectPaired rather than re-checking pairing by hand (import { expectPaired } from './transcript.js'); do the same.

Lanes: which backends run at which tier ​

packages/e2e/src/conformance/lanes.ts (CONFORMANCE_LANES) lists every (backend, tier) lane with its driver's STATIC capabilities (drivers/capabilities.ts) and a status. Expected cells are tier-aware: scenario × tier × backends with a lane at that tier. These lanes are running:

  • every Node backend at the executor tier (conformance-js / -temporal-1 and -temporal-2 / -dbos);
  • cf-do and js-chat-host at the host and client tiers (src/__tests__/conformance-cf.integ.test.ts, one file of vitest.conformance.config.ts and so one test:conformance shard in CI; pnpm --filter @helix-agents/e2e run test:conformance-cf runs it alone). js-chat-host is a host/client-only backend (harness/registries/chat-host-backends.ts), in no executor matrix;
  • cfw-workflows-d1 at the executor tier (src/__tests__/conformance-cfw-1 / -2 / -3.integ.test.ts, three files and so three test:conformance shards, each running a part of the scenarios weighted by measured cell time (src/conformance/lane-weights.ts); test:conformance-cfw runs all three). See "The CF Workflows lane" below;
  • the five store backends at the store tier (src/__tests__/conformance-store.integ.test.ts, its own config and CI job, test:conformance-store / test:conformance:store; see "The store tier" above).

No lane is declared not-yet-run today. A lane that is not ready is declared not-yet-run. Its cells are reported by conformance:verify as "not yet run": never expected (CI does not fail them as missing), and never a pass. A result line for one, a ledger fragment on one, or a provenBy naming one is a problem. Flip the lane to running in the same change that wires its entrypoint. Three guards keep the table honest:

  • a running lane with no scenario at its tier (so no expected cell) fails drift;
  • a result line for a (backend, tier) with NO lane fails conformance:verify (and the soak);
  • verifyResults takes the lane table and the not-yet-run cells as required inputs. defineConformanceSuite refuses to run a lane that is not running or whose MakeDriver capabilities differ from its lane's; the real-catalogue drift test takes each cell's capabilities from its lane, so DECLARED_GAPS stays exact per lane, running or not. (Owner-managed, like gaps.ts.)

The CF Workflows lane (cfw-workflows-d1) ​

The cfw-workflows-d1 executor lane runs the REAL CloudflareAgentExecutor in Node over Miniflare 5's Workflows binding (harness/setup-helpers/cfw-miniflare.ts, setupCfwMiniflare); workerd runs only the worker (harness/cfw/worker/cfw-conformance-worker.ts), whose ConformanceAgentWorkflow runs the product's workflow body over D1 and the StreamServer DO. Design and as-built notes: docs/superpowers/specs/2026-09-30-conformance-plan-0c-cfw-lane-design.md (§3 and §9, "As built: MR-1b" and "As built: MR-2").

  • It is running (lanes.ts) since plan 0c MR-2. Its entrypoints src/__tests__/conformance-cfw-1 / -2 / -3.integ.test.ts are three files of vitest.conformance.config.ts, one CI shard each: each runs one part of the scenarios (scenarioPart, src/conformance/scenario-parts.ts), so the lane no longer sets the pipeline's wall-clock alone (FU-CONF-SHARD-BALANCE). It needs no Docker and no service, only the build's dist/; it takes about 6-7 minutes locally. Its ledger was taken from 5 deterministic runs: every executor scenario runs on it except one declared gap (control.stale-entry-loses-after-winner; submit.race-a, the other, is retired). Every failing id is ledgered under a record, as on any lane: a new cfw failure follows §4. The lane's harness self-tests (harness/cfw/__tests__/) run in test:conformance-harness and test:conformance-unit.

  • Iterate with test:conformance-cfw, and give it its own CONFORMANCE_RESULTS_DIR, as for test:conformance-cf (§6).

  • The instance ledger and the reap. A cell session's instance ids are hashed (cell ids contain .) and cannot be parsed back into a session, so the harness never derives them. The executor reaches the engine through ObservedWorkflowBinding (harness/cfw/observed-binding.ts), which ledgers every instance the executor creates or adopts and every product terminate(). The worker's body-side binding reports each child it creates as wf.created, which the ledger folds in under its root session. At cell end the reap (terminateCellWork, harness/cfw/cell-work.ts) reads every instance of the cell's roots, gives open ones the close grace, then terminates the rest parents first. It fails closed: any status other than complete, errored or terminated counts as open. It never relabels an instance the product terminated. Terminated sessions are fenced ('reaped' or 'product-terminated'): their late hook, witness, gate and wf.created traffic is recorded in bridge.late, not a failure, through the end of the next cell. A new model.generateStep from a fenced session is still a failure. Under Miniflare 5 a terminate() also closes the instance's in-flight bridge calls, so a held model call ends connection-closed rather than staying held; see the spec's §3.6 for how that is counted. The binding's fault log is drained at endCell, and a non-empty drain fails the cell. It fails closed: every failed binding call is a fault except the engine's own answers (instance.already_exists, instance.not_found, instance.cannot_terminate, matched along the error's cause chain), so a transport error the product swallows always surfaces. Body-side terminates (the product's unknown-owner escape hatch) are ledgered too: the worker's body binding (harness/cfw/worker/observed-body-binding.ts) reports a successful terminate() as wf.terminated, and the ledger marks that instance terminatedBy: 'product', so endCell fences its session.

  • Same-key model calls. ProxyModel sends the instance's run key, so a model.generateStep that repeats an earlier call's (session, runKey, agentType, step, phase) in the cell is a re-executed step.do. The bridge tags it sameKeyAs with a cause:

    • after-retryable-error: the first call answered an error, or its model threw, with a detail that is retryable: true or has no code (an engine retry the product chose);
    • after-harness-error: a harness error of the session was recorded since the first call;
    • unexplained: anything else. Triage it first, as a potential finding, before you ledger any id of that cell.

    The cell result carries them as sameKeyCalls, for the cell's sessions and every child session rooted at one. Each retry takes the next scripted response, so script only classified non-retryable failures unless the scenario means to test the retry, and say so in the scenario.

  • A real remote child (remote-child). The lane's Miniflare runs a second worker, conformance-producer: the cf-do conformance worker, with the documented createRemoteAgentWorkerHandler mount at /agents over its own ConformanceAgentDO namespace. The cfw worker reaches it over its PRODUCER service binding (serviceBindingTransport), so a workflow parent's remote sub-agent call crosses a real binding to a real child.

    • Agents: conf-remote-parent (its one tool is subagent__delegate_remote) and conf-remote-child (a gated tool, then its typed answer). The remoteTransport port wires the parent; unwired, its transport refuses every call, so no other lane runs the child anywhere else.
    • The child runs under ${parent}-remote-${callId} (remoteChildSessionId): script it with scriptFor on that id and hold it with toolGates on that id (the bridge serves both workers).
    • Read the child with driverRemoteChild: status is the producer's answer on the remote protocol (what the parent's transport sees), persisted its DO's session. The parent's ref is in the lane's own store.
    • At a cell's end, the cell's live remote children are aborted BEFORE the bridge releases their gates, so a released child never calls the model in a later cell. A gate still held at the end is still a bridge finding.
    • Scope: it proves PARENT-side behaviour against a real remote child over a real binding. The producer is the conformance DO behind the documented mount, not createAgentServer's own wrapper (its auth, its routing). A producer-side finding needs a real createAgentServer producer.
  • settleWaits is informational. Every executor create that first waited for the session's previous instance to leave a live status is recorded in the cell result as settleWaits, derived from the binding's status-poll log. It is never a verdict: it tells a previous instance held open (for example by an unreleased model hold) apart from a product refusal. The scenario's own timings (ctx.recordTiming) sit beside it in the same result line; on this lane they show where a slow cell's time went (e.g. session.create-observed-whole's per-attempt phases).

  • Tools run in workerd. The bundled tools execute inside the workflow's step.do, so the in-process ports cannot reach them. The env supplies its own portControls (harness/cfw/bridge-port-controls.ts), and driver.toolGates / driver.toolWitnesses go through the bridge (tool.gate, tool.witness). hold-tool-gate and observe-tool-witness work as on the other executor lanes; a gate is never deleted, so a re-executed tool finds it released.

  • Per-cell agent names resolve by suffix. The worker's registry recovers a per-cell agent's cell id by matching the longest per-cell base b with agentType.endsWith('-' + b) (resolveBundledAgentType, agents/agent-type.ts). So no per-cell base may be a --suffix of another (e.g. conf-child and sub-conf-child): __tests__/agent-type.test.ts guards it, and a new per-cell entry that breaks it fails that test.

  • Never hold a tool step idle for more than about 60 s. A bundled tool held at its gate waits on its bridge request inside the workflow's step.do. Under Miniflare 5, a step left idle there for more than about 60 s stops making progress after the release: the engine resumes it only at the step's 10-minute timeout. Owner batch 33's probe of companion.wait-slice-timeout-keeps-child-live: a child released 25 s after entering its gate completed about 1 s later; one released at 65 s was still active, with no new row, 85 s after the release. So a cfw cell releases every gate it holds within about 60 s of the tool entering it, or it judges nothing that happens after the release. This is Miniflare's engine, not a product finding; whether production has a similar idle bound is FU-CFW-MINIFLARE-IDLE-STEP-EVICTION (docs/dev/follow-ups.md).

  • The engine's step log is readable (workflowInstances.runOwnerSteps). The lane runs Miniflare with unsafeLocalExplorer, and the harness reads the run owner's steps from its local explorer (/cdn-cgi/local/explorer/api/workflows/<name>/instances/<id>): each step.do, waitForEvent and sleep by name, and whether it finished. Use it to anchor a cell on the engine's own record, e.g. "the wait's waitForEvent started and has not finished" (companion.wait-paused-parent-keeps-child-live), never on a delay.

  • cfwSelfTestAccess is for self-tests only. The env's controls are read-only views (ledger, status polls, the bridge's calls and records). The raw bridge, binding and Miniflare, and with them the fault knobs (failNextWfCreated, failNextInnerCall), are reachable only through cfwSelfTestAccess(env), which lint:conformance allows only under **/__tests__/**. Scenarios, drivers and suite code may not import it.

Catalogue citations: pins are frozen ​

A record cites code as path:line or path:start-end, and every citation is one of two kinds:

  • Live: no pin. It names the code as it is on main, so when an MR moves that code, its C6 pass re-points the citation to the same lines in the new tree (byte-identical text), or pins it if the MR rewrote them.
  • Pinned: a parenthetical naming origin/main <sha> follows it, e.g. (pre-fix, origin/main 5bd30e20da), (as before WA MR2, origin/main cae966c0a7), (fix evidence, !288, as on origin/main 137b07d57a) or a bare (origin/main 4620e90228). It names the code AT THAT SHA and is frozen: no later change to the code ever moves it. The pin covers every citation in a list it ends (a.ts:1 and b.ts:5-9, :12 (origin/main X) pins all three; a semicolon ends the list, so a.ts:1; and :12 (origin/main X) pins only :12), and every citation inside the parenthetical right after it (a.ts:1 (pre-fix, origin/main X) (the guard at :7) pins :7).

Verify a pinned citation at its pin sha (git show <sha>:packages/<path>), never at a branch or at main: the code there has moved on, and a check against the wrong tree passes a moved pin. If a pinned citation is wrong at its own sha, correct it against that sha, never by following the live code.

pnpm --filter @helix-agents/e2e run conformance:pinned-refs [--base <rev>] fails when any pinned citation's numbers differ from the base revision's (default origin/main; for a C6 commit, the revision its records started from). A correction made against the pin passes with --allow <record id,…>. Run it before handing over any commit that edits catalogue citations. CI runs it in conformance:verify on every MR and merge-train pipeline, against what the MR lands on: in a merged-result or merge-train pipeline that is HEAD^1 (the target, or the car ahead, with every earlier correction already in it), and in a detached MR pipeline the MR's diff base. There a correction is declared with a Pinned-Ref-Correction: CP-71, RM-70 line (--allow-from-trailers), read from the trailers of the commits base..HEAD AND from the MR description ($CI_MERGE_REQUEST_DESCRIPTION). Put it in the MR description as well as on the commit: MRs merge squashed, and the merge-train pipeline runs on the squash commit, whose message is the MR title alone (GitLab ignores a squash_commit_message there), so a trailer on a branch commit never reaches it, while the description reaches the job as CI_MERGE_REQUEST_DESCRIPTION (!357's first train run failed exactly so). Put the line at the TOP of the description: GitLab keeps only the first 2700 characters in CI_MERGE_REQUEST_DESCRIPTION, and the gate names the cut when a truncated description declares nothing. The parser and its pin rules are src/conformance/pinned-refs.ts.

Why: a C6 pass in October 2026 re-pointed 234 pinned citations along with the live code they once matched. Its check compared each one's text at the base and at the branch, never at the pin, so every one passed.

4. Ledger rows come only from observed reality ​

Never predict a ledger fragment. First write the scenario with your finding still pending (or unledgered), run the matrix (§6), and read the actually-failing assertion ids per cell from the results JSONL ($CONFORMANCE_RESULTS_DIR/results-*.jsonl) or the conformance:verify output. :crash and :unevaluated (the synthetic ids) are never ledgerable — a crash means the scenario itself broke, not that the finding reproduced; :unevaluated means the positive path never checked something, a scenario bug, not a defect.

Then, for every failing id, walk this decision order:

  1. It's your finding → add a ledger fragment to your own record for exactly this { scenario, tier, backends, assertions }; delete pending.
  2. An EXISTING open finding FULLY explains it → propose a fragment in THAT finding's record to the conformance owner (§1), and add it only with their agreement. Worked example: CP-62's stop.mid-tool-ignores-abort|dbos-postgres|executor cell fails on three ids from three separate defects — .transcript-paired is CP-62 itself (the missing tool abort), .outcome-interrupted is CP-57's pre-existing DBOS status-misreport bug, and .continuation-completes is CP-19's pre-existing durable-interrupt-flag bug. Each id's fragment lives in the record that actually explains it (catalogue/control-plane/CP-57.ts, CP-19.ts, CP-62.ts), cross-referenced via each record's evidence (related: CP-57, etc.).
  3. Otherwise → it's a new finding. Send the owner: title, file:line evidence, and the repro cell (scenario|backend|tier). Never ledger it under your own finding id to make CI pass.

5. The fix lifecycle ​

The three steps from catalogue/README.md:

  1. Reproduce — add a scenario (a new file, or an assertion in an existing one under §1's controller ruling), run the matrix, add ledger fragments for exactly the observed failing ids (§4), delete pending.
  2. Fix — make the cells pass; delete the fragments; set status: { kind: 'fixed', provenBy: [<every cell key that now proves it>] }, naming each proving scenario (or one of its assertion ids) as a whole token in a conformance: evidence note (see below).
  3. Run pnpm --filter @helix-agents/e2e run conformance:docs and commit the regenerated docs/conformance/findings.md. It also writes docs/conformance/status.md, the burn-down, which is NOT committed (.gitignore): it is all aggregates, so it would conflict across every parallel catalogue MR.

Never delete a record file. A finding that no longer applies (refuted, or resolved-not-a-defect) is retired via status: { kind: 'withdrawn', reason } — never by removing catalogue/<area>/<ID>.ts. checkCatalogue pins the EXACT number of findings per prefix (CATALOGUE_SIZES) and requires each prefix's ids to be exactly 01..N, so a deleted record fails CI loudly instead of silently shrinking the catalogue. Only the conformance owner edits CATALOGUE_SIZES, in the same change that adds a finding.

What checkDrift (drift.ts) enforces around step 2, so "flip to fixed" is never partial:

  • A fixed finding's provenBy is non-empty, and every entry parses as a CellKey (`${scenario}|${backend}|${tier}`) naming a real scenario, a real backend, and a tier the scenario actually runs at.
  • None of a fixed finding's provenBy cells may still appear in the merged LEDGER — a stale fragment left behind after flipping to fixed fails drift ("X is fixed but <cell> is still ledgered").
  • A fixed finding's provenBy scenarios must be reachable and relevant: no provenBy key may name a DECLARED_GAPS cell (it never runs, so it can't prove anything), and every provenBy scenario id must appear in one of the finding's own evidence strings prefixed conformance: — when you flip a finding to fixed, name the proving scenario(s) in a conformance: evidence note (see catalogue/README.md's lifecycle step 2).
  • open ⇒ exactly one of ledgered or pending: an open finding has either ≥1 ledger fragment or a pending entry, never both and never neither. You cannot leave a finding half-migrated (fragments deleted, pending not re-added, or vice versa).

Separately, checkPendingPlans (plan-rule.ts) — not checkDrift — enforces that every pending entry's plan (0b/0c-do/0c-cfw/0d) agrees with plan-rule.ts's mechanical derivePendingPlan(finding.evidence), or carries a non-empty planRuleException saying why the rule is overridden. Plan 0c is split by Cloudflare backend: Cloudflare-only evidence derives 0c-do when any of it is Durable Object code (DOAB:, runtime-cloudflare/src/do/, runtime-cloudflare/src/do-clients.ts, runtime-cloudflare/src/__tests__/do-*, ai-sdk/src/cloudflare/), and 0c-cfw (the CF Workflows lane) when it is only the rest of runtime-cloudflare/ or store-cloudflare/. Every evidence item must be placeable: a packages/-relative path, the DOAB: abbreviation, or a note prefix (conformance: / related: / contrast: / live: / inferred: / severity / not yet reachable) — an item the classifier can't place fails that check.

6. Running locally ​

The commands below assume this repo's shared-sandbox containers, helix-conf-redis / helix-conf-pg (ports 6390 / 5433 — non-default, so they don't collide with other projects' containers that may already be bound to 6379/5432/7233 in a shared sandbox). If your environment has nothing on the defaults, docker-compose.test.yml's own ports work too — see docs/dev/development-workflow.md's "Runtime conformance suite" section for that path:

bash
docker start helix-conf-redis helix-conf-pg   # ports 6390 / 5433
export REDIS_URL=redis://localhost:6390
export POSTGRES_URL=postgresql://test:test@localhost:5433/helix_agents_test
export HELIX_TEMPORAL_BACKEND=test-server
export CONFORMANCE_RESULTS_DIR=$PWD/packages/e2e/.conformance-results
rm -rf "$CONFORMANCE_RESULTS_DIR"
pnpm run build   # unit tests need dist for core/store-memory/workspace-memory

Typecheck harness and scenario code BEFORE you ask for a heavy slot: pnpm --filter @helix-agents/e2e exec tsc --noEmit -p tsconfig.json against an existing build's dists is one package, takes seconds, and catches what vitest never does (it strips types). A slot that starts on untypechecked code can end at its own typecheck step and give its hold back unused (owner slot s50: four type errors in a new capability's plumbing, 2026-10-19).

Then, from the repo root:

bash
# No infra needed (but needs the build above): src/conformance/__tests__,
# scripts/__tests__, the cf-do harness's unit tests (src/harness/cf-do/__tests__/*.test.ts)
# and the cfw harness's (the binding wrapper, the reap, the port controls)
pnpm --filter @helix-agents/e2e run test:conformance-unit

# The cf-do and cfw harnesses' Miniflare self-tests (spike, fingerprint drift,
# persisted reads, timing controls, driver live smokes; for cfw also the reap,
# the fences, the same-key causes and the lane live smoke). No Docker; needs the build.
pnpm --filter @helix-agents/e2e run test:conformance-harness

# Type-aware no-floating-promises lint over src/conformance, src/harness, and
# the conformance-*.integ.test.ts entrypoints — catches an un-awaited check()/settle().
pnpm --filter @helix-agents/e2e run lint:conformance

# The Node matrix (js-*, temporal-*) and the DBOS matrix (needs RUNTIME_DBOS_INTEG=1
# additionally set for DBOS-integration coverage elsewhere; the conformance
# DBOS entrypoint itself does not gate on that var).
pnpm --filter @helix-agents/e2e run test:conformance
pnpm --filter @helix-agents/e2e run test:conformance-dbos
# test:conformance includes the host and client tiers (cf-do under Miniflare +
# js-chat-host) and the CF Workflows executor lane (Miniflare). To iterate on
# either ALONE (no Docker or services; needs the build above — the workers'
# bundles resolve @helix-agents/* from dist/):
#   pnpm --filter @helix-agents/e2e run test:conformance-cf
#   pnpm --filter @helix-agents/e2e run test:conformance-cfw
# Each writes results too, so give it its own CONFORMANCE_RESULTS_DIR: a cell
# that ran twice into one dir (here and in a full test:conformance) fails
# conformance:verify.
# The store tier (its own entrypoint, not in test:conformance; needs the build
# and docker-compose.test.yml's Redis + Postgres for the store-redis /
# store-postgres lanes):
pnpm --filter @helix-agents/e2e run test:conformance-store

# Locally this ONLY merges results against the ledger — the unit/lint/docs-check
# gates are wrapped around it in the CI job's script (§7), not inside the
# script itself. Run test:conformance-unit / lint:conformance / the
# --check command below yourself before trusting a green conformance:verify.
CONFORMANCE_RESULTS_DIR=$PWD/packages/e2e/.conformance-results \
  pnpm --filter @helix-agents/e2e run conformance:verify

# Regenerate findings.md (committed) and status.md (local only) after editing a catalogue record.
pnpm --filter @helix-agents/e2e run conformance:docs
# Check-only (what CI runs; fails if the checked-in findings.md is stale):
pnpm --filter @helix-agents/e2e exec tsx scripts/conformance-docs.ts --check

To run one scenario while iterating, add a vitest -t (test-name) filter — each cell's test name is its `${scenarioId}|${backend}|${tier}` key, so target one scenario on one backend with:

bash
pnpm --filter @helix-agents/e2e run test:conformance \
  -t 'transcript\.finish-cosiblings\|js-memory\|executor'

(| is a regex alternation operator, so escape the literal pipes in the key.) A -t-filtered run only excuses the cells it filtered out of the reporter's "every marked cell must have a result" check — it never counts as a pass for them, so a filtered run is for local iteration only, never a substitute for the full matrix.

The cf-do backend keeps each Miniflare's DO-SQLite under a short literal /tmp/hxc-XXXXXX dir (harness/cf-do/miniflare-backend.ts), because macOS's os.tmpdir() path is long enough to overflow workerd's persistence paths. Each backend removes its dir on dispose; a killed run can leave /tmp/hxc-* dirs behind, safe to delete when no run is active. Its cells need nothing else: Miniflare runs workerd in-process, with no Docker.

7. CI ​

ci/test-matrix.yml: test:conformance (sharded parallel: 7, matching vitest.conformance.config.ts's literal include file count — ci-guard.ts fails loudly on drift; the files are the js lanes plus the canaries, the Temporal lanes in two parts, the host/client-tier conformance-cf.integ.test.ts and the CF Workflows executor lane in three parts. A lane split across parts runs scenarioPart(ALL_SCENARIOS, k, n) in part k; conformance:verify catches a cell missing from every part or run in two. The cf and cfw files need no service, only the build job's dist/ artifacts that needs: [build] downloads), test:conformance:dbos and test:conformance:store (the store tier, using .integration_base's Redis and Postgres services) run the matrix; conformance:verify (needs: [build, test:conformance, test:conformance:dbos, test:conformance:store], rules: ... when: always, so it still runs even when a shard failed) merges every shard's results and runs, in order, collecting every exit code rather than stopping at the first failure: test:conformance-unit → test:conformance-harness → lint:conformance → conformance-docs.ts --check → conformance:verify. The harness step is the cf-do and cfw Miniflare self-tests: they guard the cell contract, so they run here, never in the test:integ:e2e job (vitest.config.ts excludes src/harness/cf-do/** and src/harness/cfw/**). ci-guard.ts's checkConformanceGateJob fails verify if the job drops one of these steps or starts retrying on script_failure. The job fails at the end if any step failed. The "Conformance report" artifact (packages/e2e/.conformance-report/summary.md) is published on the merge request regardless of which step failed first.

Adding a new matrix entrypoint file means bumping test:conformance's parallel: count in ci/test-matrix.yml to match include's file count — this is harness-owner territory (§1), noted here because ci-guard.ts enforces it.

8. Flake policy ​

No automatic retries inside the suite. Conformance jobs never retry on script_failure (no CI job does) — a rerun must never paper over a real ledger mismatch. They do retry once on runner_system_failure/unknown_failure (genuine pipeline/runner infrastructure problems — a pod watcher error, a service dying — not suite behavior). If a shard dies for an infra reason, conformance:verify fails with "ledger entry never ran" (a missing result line, not a mismatch); retry conformance:verify once the shard itself has passed, rather than trying to interpret the stale merge. The nightly conformance:soak job runs the full matrix three times and fails on any cell with mixed verdicts across runs, a missing result in any run, or any run failing conformance:verify's CI-mode checks — a cell failing the same way in every run is not flaky and still fails the soak.

Timing bounds (STOP_ABORT_BOUND_MS, HELD_BOUND_MS, the 500ms MID_TOOL_RELEASE_DELAY_MS in stop.ts, etc.) carry documented headroom — never widen a bound just to make a cell green without first measuring why it's slow. A scenario with a timing-dependent assumption must say so in a doc comment: stop.ts's midToolScenario states, right above MID_TOOL_RELEASE_DELAY_MS's use, that releasing the abort-ignoring tool 500ms after stop() is requested (concurrently with it, not after it returns) is a best-effort assumption — it only guards against a runtime that never delivers the tool's abort signal at all, and a slower interrupt delivery than 500ms degrades that cell to "stop after the tool returned" rather than failing outright.

Held work must always release ​

Any scenario that gates a running tool or a held model call must release it in a finally, unconditionally — see stop.ts's midToolScenario:

ts
} finally {
  // MANDATORY (integrity): always let the tool finish, so a runtime
  // that never delivers the abort can still drain its run and the
  // backend rebuild after an unsettled cell can dispose it.
  clearTimeout(releaseTimer);
  gate.release();
  // The gate is deliberately NOT deleted: a late re-execution of `slow`
  // (a Temporal activity retry, DBOS step recovery after a rebuild)
  // must find it already released, not a fresh gate nobody releases.
}

The same applies to the timing controls: release every host-hook hold and tool gate a scenario places in a finally. endCell names any hold still in effect (a crash, never a hang), and a disposed driver releases and names the ones it placed (see "Timing controls" above).

A runtime that never delivers an abort must still be able to drain its run — otherwise the backend rebuild after an unsettled cell (§9 of the design spec) can't dispose the stuck instance, and the backend gets poisoned (GATED (<reason>) for every later cell on it, hiding their real verdicts). Gates are keyed by session id and are never deleted after release — a retried tool execution (a Temporal activity retry, a DBOS step replay) must find its gate already released, not register a fresh one nobody releases.

Temporal test-server: hold on a tool latch, not a scripted model call ​

Harness hazard H1. With HELIX_TEMPORAL_BACKEND=test-server, the Temporal lanes run on the in-process Java TestWorkflowEnvironment.createTimeSkipping() server. On that server, time skipping is active whenever ANY run's workflowHandle.result() is being awaited, and the executor driver awaits one for every run it tracks (_temporal-client-adapter.ts). While time is skipped, server-side timers and activity timeouts advance faster than the wall clock.

So a scenario that holds a scripted model call (a { kind: 'hold' } script entry) across a long wait can see the model call's activity time out on the server while it is still held in the harness. Temporal then retries the activity, and the retry consumes the NEXT scripted response. The cell ends up judging a transcript the scenario never scripted, and the failure looks like a product defect.

  • Prefer an idempotent tool latch for any long hold: a tool that marks a gate reached and waits for the scenario to release it, keyed by session id and never deleted, like stop.ts's slow tool (toolGate) or MR !270's warmup latch. A retried tool execution finds the latch already in the state the scenario left it in, and it consumes no scripted model response.
  • Keep scripted-model holds short, bounded to reaching the hold and acting on it (e.g. awaitCallHeld followed by stop()), not a wait on other work.
  • The Docker dev server (HELIX_TEMPORAL_BACKEND=docker, CI's backend) does not skip time, so a scenario that is green there can still hit H1 on the test-server. Run a new Temporal scenario on both before ledgering anything.
  • A run held at a run-start hold is exposed too: the skipped clock can fire the workflow's execution timeout while it is held. Known case: stop.drain-lapse-after-stop fails intermittently on the test server with Workflow execution timed out and is green on Docker (owner slot s96, 2026-10-23). It has no gap or ledger on Temporal, since CI runs Docker.

Scenario-design artifacts are not defects ​

Before ledgering an observed failure, ask whether the scenario itself forces a contract the spec doesn't actually state. Example: releasing a held tool only after stop() returns would deadlock any runtime whose interrupt() implementation waits for the run to end before resolving — that's a property of the scenario's timing choice, not a runtime defect. stop.ts's midToolScenario releases the tool on a timer started when stop() is requested, concurrently with awaiting it, specifically to avoid manufacturing that deadlock. If you find a scenario forcing an unstated contract, that's a question for the bucket's own contract spec / owner — not something to ledger as a finding.

Released under the MIT License.