One result per remote call (remote settlement)
This guide covers the release in which a remote sub-agent's ref and its call's result are committed together (Invariant S), and in which the parent settles its remote children by one transitions table (RM-98, RM-99, CP-176, CP-177, CP-178, RM-100, RM-101). It is a minor release of @helix-agents/core (a 0.x package: minor marks an observable behaviour change), of every store package (store-memory, store-redis, store-postgres, store-cloudflare, and the CF Durable Object store in runtime-cloudflare) and of @helix-agents/runtime-js, @helix-agents/runtime-cloudflare and @helix-agents/runtime-temporal.
No migration and no deploy drain. No column, workflow input or instance param changed: the settlement uses the ref columns every store already has (status, completion time and outcome generation).
What was wrong
- A call could get two results (RM-98, RM-99). The parent wrote a remote child's result and its ref in two separate writes. A crash or a failed write between them left the call paired while the ref still read
running, and the next resume paired the call again. A store error during reconcile was read as "the child is lost", which appended a second, failed result. A duplicatetool_resultis a provider 400 on every later model call, andretry()keeps it, so the session was wedged. - Children were settled by too few rules. A child still running past every bound stayed
runningforever (CP-176). A child whose/startwas still committing read "not found" and was settled lost at once (CP-177). A lost child was never stopped (CP-178). A transient status-read failure on a new turn orphaned a child that might have finished (RM-100). A ref whose remote tool was renamed or removed was skipped forever, leaving its call unpaired (RM-101).
The new rule
A commit that writes a remote call's result can carry a settle list (MessageCommit.settle, SaveAndPromoteOptions.settle): the store settles those refs inside the same commit, and refuses the whole commit if one is no longer live. Entries and reconcile on runtime-js / the CF DO, CF Workflows' entries and result appends, and recordSubSessionResult (CF Workflows, Temporal) use it. runtime-js's in-run dispatch still writes the ref terminal just before its step commit (a crash in between leaves the call unpaired, and the next resume pairs it with the child's real outcome), and DBOS does not settle atomically yet (below).
| Error | Code | Retryable | HTTP |
|---|---|---|---|
SubSessionRefConflictError | state_sub_session_ref_conflict | no | 409 |
SubSessionRefMissingError | state_sub_session_ref_missing | no | 409 |
RunStartRejectedError('sub_session_ref_conflict') | state_run_start_conflict | re-planned once | 409 |
RemoteChildStatusUnavailableError | framework_subagent_status_unavailable | yes | 503 |
The parent decides each child's fate from the transport's getStatus answer alone (core remoteChildVerdict). The child's deadline is startedAt + the runtime's watch budget (its attempts and the retry waits between them) + 60 s. The full table is in the remote agents guide.
What your callers see
- A new turn (or a
with_messageresume, or a retry with a message) can be refused with a retryable 503 (framework_subagent_status_unavailable) while a remote child's status cannot be read inside its deadline. Nothing is written; send the message again later. A stockuseChatputs it inchat.error;parseHelixChatError(chat.error)reads the code andretryAt, the epoch ms after which the child is settled lost and the entry proceeds. The window is the runtime's watch budget + 60 s from the child's start, by design (a child is never declared lost while it may still run); shorten it with the remote tool'stimeoutMs/streamRetries/streamRetryBaseMs. - A lost child is now interrupted (best-effort
transport.interrupt()), and its call gets the typedframework_subagent_lostresult.causegainsdeadline_exceededandtool_unavailable. - An
interruptedremote child is settled as a typed failure on resume, instead of being leftrunning. - A child whose producer does not know it yet is still awaited until its deadline, unless its start was confirmed (the ref records the child's stream).
- CF Workflows keeps a replaced (too-large) child's true
completedstatus on its ref at an entry, as runtime-js and the CF DO do.
Removed: core's lostRemoteChildDetail(error, ref, timeoutMs, now). Use remoteChildVerdict(read, ref, config, now), which takes the transport read's outcome (a status or an error) and returns the whole verdict.
If you implement a store
Apply settle inside the commit's atomic step with core settleRefusal(sessionId, settle, actual), where actual is the ref's status and outcomeGeneration as the commit reads them, on appendMessages (append and run-start commits) and saveStateAndPromoteStaging. Run the contract suite: subSessionRefOperationTests (part of stateStoreContractTests).
If you write your own loop
Pass settle: remoteSettlementsFor(rows, await store.getSubSessionRefs(sessionId), Date.now()) on any commit whose rows carry a remote child's result. Treat a SubSessionRefConflictError as "another writer recorded this call's result": do not append it again. Read a remote child's fate with remoteChildVerdict, never from a store error. assertToolResultPairing(messages) checks a transcript has one result per call.
Not yet on DBOS and Temporal
DBOS writes a remote child's terminal ref before its workflow appends the result (FU-DBOS-REMOTE-SETTLE-ATOMIC). Temporal and DBOS heal a still-running remote child's trailing call not_executed at the next entry without settling its ref (FU-TEMPORAL-REMOTE-HEAL-SETTLE). Temporal's recordSubSessionResult path settles atomically.