Skip to content

Interrupt and Resume ​

Helix Agents supports interrupting and resuming agent execution. This enables user-controlled pauses, crash recovery, time-travel debugging, and human-in-the-loop (HITL) workflows.

Overview ​

When an agent is running, you can:

  • Interrupt — soft stop that saves state for later resumption.
  • Resume — continue execution from where it stopped.

This is different from abort, which is a hard stop that fails the agent permanently.

v7 changes at a glance ​

Three things changed in v7 that touch this guide directly:

  1. Durable interrupt protocol. v6's interrupt(handle) walked an in-memory activeHandles map and returned 503 INTERRUPT_NOT_LOCAL when the handle wasn't on the same process. A stop is now a durable record on the run (see Durable stop protocol); the running loop picks it up at its next boundary regardless of which process owns the run. The 503 response is gone.
  2. Three new suspended statuses. RunOutcome.kind (and therefore handle.result().status) gains 'suspended_client_tool', 'suspended_awaiting_children', and 'suspended_step_partial'. Existing switch statements that only handle 'completed' | 'failed' | 'interrupted' will fall through silently for HITL agents.
  3. Resume is explicit for HITL. When a run suspends at a HITL boundary, submitToolResult no longer wakes an in-memory waiter. The chat handler (or your code) must call executor.resume({ sessionId }) after the submission lands.

If you are upgrading from v6, read the v6 to v7 migration guide end-to-end before deploying. The rest of this page describes the v7 model.

execute() vs resume() Semantics ​

Understanding the distinction between execute() and resume() is critical for correct agent lifecycle management.

execute() - Fresh Start ​

execute() always starts a fresh execution. Using the same sessionId continues the conversation within that session:

typescript
// New session with generated ID
const handle = await executor.execute(agent, 'Hello');

// New session with specific ID
const handle = await executor.execute(agent, 'Hello', { sessionId: 'my-session' });

// Continue conversation in existing session
const handle2 = await executor.execute(agent, 'Follow up', { sessionId: 'my-session' });

When you call execute():

  1. New run begins - A run-start commit appends your input and creates the turn's run record in one atomic step. It is the only gate: a concurrent caller that loses it writes nothing.
  2. Stream continues - The session's stream is made live again for the new run. Earlier chunks are kept; the new run's chunks follow its run_started marker.
  3. Session state continues - Messages and custom state are preserved within the session

This means each execute() call starts a fresh run. If you call execute() with the same sessionId, the new run continues the conversation with access to previous messages.

resume() - Continue Execution ​

resume() continues from where the agent stopped:

typescript
// Continue from last checkpoint
const newHandle = await handle.resume();

// Continue with additional context
const newHandle = await handle.resume({
  mode: 'with_message',
  message: 'Please continue',
});

When you call resume():

  1. New run begins - Every resume() is a new run with its own run-start commit, which supersedes the suspended or interrupted run
  2. Stream is preserved - Existing stream chunks remain intact
  3. State is loaded - Agent state is restored from the checkpoint
  4. Execution continues - The agent resumes from the checkpoint step

When to Use Each ​

ScenarioMethodReason
Starting a new conversationexecute()Fresh start needed
User sends new message to completed agentexecute()New run, new stream
Crash recoveryresume()Continue where left off
User interrupted, wants to continueresume()Preserve progress
Time-travel to earlier stateresume({ mode: 'from_checkpoint' })Branch from history
Tool confirmation receivedresume({ mode: 'with_confirmation' })Continue paused run
Retry after failureretry()Restore checkpoint, re-attempt

Stream Behavior Difference ​

The key technical difference is stream handling:

typescript
// execute() starts a new turn on the session stream
// - Clients see: the new run's chunks after its run_started (no stream_resync)
// - Earlier turns' chunks stay in the stream

// resume() continues the turn
// - Clients see: existing chunks → new chunks from the checkpoint step
// - A from_checkpoint rewind also writes one stream_resync

The chat handler reads the stream head before it calls execute() and streams only what follows it, so a new turn never replays the previous turn's chunks.

This distinction matters for frontend integration. If you're building a chat UI:

  • New conversation: Use execute() - the UI should clear and show fresh messages
  • Reconnecting after disconnect: Use resume() - the UI should show existing messages and continue

Retrying Failed Agents ​

Failed agents cannot be resumed with resume(). Use the retry() method for failure recovery.

Why retry() is Separate from resume() ​

Aspectresume()retry()
PurposeContinue interrupted/paused executionRecover from failure
Valid statusesinterrupted, pausedfailed
Stream behaviorPreserves chunksWrites a stream_resync
State handlingContinues from currentRestores from checkpoint

retry() Method ​

typescript
const result = await handle.result();
if (result.status === 'failed') {
  // Retry the failed run from its last committed step, with a message to
  // continue with. (A run that failed on its first step can also be retried
  // with no message: its own triggering message is re-sent.)
  const retryHandle = await handle.retry({
    message: 'Please continue',
  });

  // Or retry from a specific checkpoint
  const retryHandle = await handle.retry({
    message: 'Research quantum computing',
    checkpointId: 'cpv1-...',
  });
}

Every retry() is a new run. Its run-start commit rewinds the log to the retry target (StartRun.truncateTo), appends the retried input and creates the run, in one atomic step, on every runtime. A concurrent retry() that loses the run start writes nothing, and so does a stale one: a retry() planned before another entry's run finished (or before the session was aborted) is refused in the commit instead of rewinding that run's work (AgentAlreadyRunningError, or RunStartRejectedError('entry_status_changed'); see Distributed Coordination).

Which checkpoint a retry restores. A failed run ends with a terminal state commit, like every run, so the session's latest checkpoint after a failure is that terminal commit. retry() never restores it. Core planRetry picks the target on every runtime:

  1. The failed run's last committed step. The run's latest step commit, or a later append commit of that run that carries no assistant message (for example a finishWith result appended after its step). The retried run continues from there, with options.message as its input (after a maxSteps cut-off, no message is needed: see below).
  2. Otherwise, the run's run-start commit. The run failed on its first step. The log is rewound to just before the run's own triggering user message, and the retried run appends that message again, followed by options.message when you pass one.

The restored customState, stepCount and forced-completion state come from that target. An explicit checkpointId is restored as given: the log is cut to its messageCount, and the retried input is options.message, or else the last user message after that checkpoint.

Message Requirement

Without message, retry() retries with the last user message after the restored target. That message exists only when the run failed on its first step: the rows were rewound to just before the run's own triggering message, so the trigger is re-sent. In every other case there is no user message after the target, and retry() without message throws validation_error (No user message found to retry with. Provide a message in options.):

  • the run committed one or more steps before it failed (the target is its last step), unless it failed with framework_max_steps_exhausted (below);
  • the session failed before its last entry's run-start commit (for example, the entry failed before it committed). Nothing of that entry reached the log, so the retry keeps the whole log;
  • an explicit checkpointId has no user message after it. This includes the failed run's own run-start checkpoint (kind input): its rows already end at the run's triggering message, so nothing is left after it to re-send. Restore it with message. The default retry of a first-step failure rewinds to just before the trigger, so it re-sends the trigger itself.

Pass message when in doubt. On runtime-js, retry() with no message used to throw after a first-step failure; it now re-sends the trigger (CP-58). DBOS used to ignore RetryOptions; it now honours checkpointId and message like every other runtime (CP-30). If the session has no checkpoint at all, retry() restarts fresh from the triggering message.

Temporal: rewinding to a finishWith step's checkpoint deletes the call's result (RM-52)

Temporal commits a step that calls a finishWith tool before it appends that call's result, which lands in a separate append commit. The default retry target is that later commit, but a retry({ checkpointId }) or resume({ mode: 'from_checkpoint' }) to the step's own checkpoint truncates the transcript to it, which leaves the finishWith call's tool_use unpaired. The JS runtime commits the step with its result. On Temporal, do not rewind to the checkpoint of a finishWith step. See RM-52 in the conformance findings.

Retry after a maxSteps cut-off ​

A run that ran out of maxSteps fails with framework_max_steps_exhausted (see When maxSteps runs out), and its run record says completionReason: 'max_steps'. retry() treats it as unfinished work, not an error:

  • it restores the run's last committed step (rule 1 above), keeping its log, customState and forced-completion state;
  • it restarts stepCount at 0, so the retried run gets the full maxSteps budget, even when the cut-off run was itself a resume that entered with a higher count;
  • it needs no message: the kept log ends in the last step's tool results, so the model carries on. A message you pass is appended as new input.

This also holds when the cut-off run committed no step. For example, a client tool called on the last allowed step suspends the run; the resume then enters at the cap and is cut off before it calls the model. retry() keeps that resume's entry, including the submitted client-tool result, and continues from it with a full budget and no message.

An explicit checkpointId is still restored as given. This holds on JS, the Cloudflare DO and Cloudflare Workflows. Temporal and DBOS do not fail a cut-off run yet (DI-25), so there retry() refuses it as not failed (FU-CFD1C-DI29-DI25-TEMPORAL / -DBOS).

RetryOptions ​

OptionTypeDefaultDescription
checkpointIdstring(see above)Which checkpoint to restore from. Default: the failed run's last step or its run-start commit. An unknown id throws state_checkpoint_not_found; another session's checkpoint throws state_checkpoint_foreign. Both are rejected before any state change.
messagestring | UserInputMessage[](extracted)Message to retry with. Default: the last user message after the target (the failed run's trigger after a first-step failure). Not needed after a maxSteps cut-off. Required otherwise.
abortSignalAbortSignal-Abort signal for cancellation

Concurrency Safety ​

One run at a time: the run-start commit ​

Every execute(), resume() and retry() call is an entry that starts a new run. Its run-start commit is the single gate, on every runtime. In one atomic step the store checks that the current run is still the one the entry saw, refuses while another live run owns the session, supersedes the previous run, applies the entry's rewind (if any) and creates the new run with its input rows. A caller that loses writes nothing: no input row, no stream change, no chunk.

A loser gets a typed error:

ErrornameMeaning
RunStartConflictError (extends AgentAlreadyRunningError)AgentAlreadyRunningErrorA live run owns the session (conflictCause: 'run_active'), or a newer run started since this call read the session ('expected_mismatch'). Wait, or attach.
RunStartRejectedErrorRunStartRejectedErrorThe commit was refused for another reason (rejectCause: idempotency_mismatch, entry_status_changed (the session changed under the entry: reload, then retry), entry_abandoned). Do not treat it as "already running".

Both carry code: 'state_run_start_conflict'. A refusal before the commit (for example assertNoLiveOwner on JS) throws a plain AgentAlreadyRunningError, so check with instanceof AgentAlreadyRunningError:

typescript
import { AgentAlreadyRunningError } from '@helix-agents/core';

try {
  await executor.execute(agent, message, { sessionId });
} catch (error) {
  if (error instanceof AgentAlreadyRunningError) {
    console.log(`Session ${error.sessionId} is already running`);
    // Wait for the existing run to finish, then try again
  }
}

Who counts as a live owner depends on the runtime: on JS, a run whose lease (JS_RUN_LEASE_MS, 30 s, renewed every 10 s) has not lapsed; on Temporal and DBOS, a run whose workflow is still open; on Cloudflare Workflows, a run whose instance is live; on the Cloudflare DO, the execution in progress in the object. A run whose owner is gone (a crashed process) is superseded by the next entry. See Distributed Coordination.

Superseded runs ​

When an entry supersedes a run, that run's record turns superseded, and every later write the old executor attempts is fenced: commits, saveState, run status changes, stream chunks once the new run has claimed the stream, and tool executions all fail with RunSupersededError (state_run_superseded, never retried) and write nothing. The old handle's result() settles with status: 'superseded' on every runtime (on runtime-js with errorDetail.code: 'state_run_superseded'). onAgentSuperseded fires for it and onAgentFail never does. onAgentComplete does not either, unless the supersede landed after it had already fired and before the run's terminal commit: then the run fires onAgentComplete then onAgentSuperseded, and the last hook is authoritative (see Hooks).

typescript
const result = await handle.result();
if (result.status === 'superseded') {
  // Another run took over this session. Its output is in the session, not here.
}

A superseded run is not a failure, and the session is not failed because of it. On runtime-js an event loop blocked for longer than about 20 s (two thirds of the lease) can let another entry supersede the run; the stopped run reports superseded.

resume() against a live run is refused on every runtime (CP-83)

The resume CAS accepts active on purpose: a session awaiting client-tool submissions can read active (on Temporal, a session re-suspended on a partial client-tool batch does), and resuming it is correct. A session that reads active with no pending client-tool or approval calls is a live run, and resume() refuses it in every mode with AgentAlreadyRunningError, writing nothing: continue and with_message by that check, from_checkpoint by the rewind liveness gate. DBOS and the Cloudflare DO (409 while executing) always refused; Temporal, Cloudflare Workflows executor.resume() and JS from_checkpoint used to accept the live run and supersede it, and its answer was lost. Cloudflare Workflows handle.resume() had its own copy of the resume path without the check; it now runs executor.resume(), so the same guard applies. See CP-83 in the conformance findings.

Basic Usage ​

Interrupting an Agent ​

Use handle.interrupt() to pause execution:

typescript
const handle = await executor.execute(agent, 'Research quantum computing');

// Later, interrupt the agent. This resolves once the stop is recorded;
// it does not wait for the run to end.
await handle.interrupt('user_requested');

// The run ends at its next boundary
const result = await handle.result();
console.log(result.status); // 'interrupted'

The agent will stop at the next safe point (between steps). Any in-progress step is not committed: it is rolled back to the last checkpoint. A run suspended on client tools is stopped at once.

The rollback truncates only rows past the latest checkpoint, and since every message row is written by a commit, the latest checkpoint counts every committed message: a turn's input is part of its run-start commit, and each step's rows are part of its step commit. An interrupt therefore never deletes the turn's input or an earlier step (RM-38, RM-41, RM-42). A stop that lands before a step's commit makes the commit refuse atomically (CommitStoppedError), and the step is discarded (on DBOS with a step_discarded chunk).

DBOS: an interrupt right after execute() keeps the turn's input (CP-79)

execute() on DBOS now returns its handle only after the workflow's run-start commit, which persists the input together with the run record. An interrupt through that handle, however soon, can no longer end the session interrupted with no user message. Earlier releases persisted the input several durable steps into the workflow, so an interrupt that cancelled the workflow first lost it. The handle's result() after such an interrupt is tracked as CP-57. See CP-79 in the conformance findings.

Durable stop protocol ​

A stop (interrupt or abort) is a durable record on the run it targets, not an in-memory signal. v6 looked the handle up in an in-memory activeHandles map and returned HTTP 503 INTERRUPT_NOT_LOCAL if the handle lived on a different process. Now every stop goes through core stopRun (WA MR2):

  1. The stop is recorded on the run (requestStop). It names that run, so a stale stop never stops a newer run, and nothing clears it.
  2. If an owner is executing the run, it is signalled and the call answers stopping (HTTP 202). The run ends at its next boundary.
  3. If nothing executes the run (it is suspended on client tools, or its process is gone), the stop's terminal is committed now: the answer is interrupted / failed (HTTP 200).
  4. With no live run, nothing is written: not_stoppable (HTTP 409 state_not_stoppable).

The call answers at once and never waits for the run to wind down: observe the end on the stream (run_interrupted) or through the status. So POST /chat/{id}/interrupt (or POST /interrupt on the executor route) succeeds from any process, with no sticky routing and no observation deadline (agent-server's interruptObservationDeadlineMs and its 504 are gone). An abort ends the run failed with framework_cancelled; an abort that arrives after an interrupt upgrades it. Handles' interrupt() / abort() resolve once the stop is durable. See the upgrade guide for the answers per runtime.

On the Cloudflare DO, a run that ignores the stop (a tool that ignores its abortSignal) is ended by the stop deadline (stopDeadlineMs, default 30 s). The other runtimes have no deadline yet.

Stopping from the chat UI (CP-12) ​

A chat client's Stop button reaches the server only through a stop route. Wire one, then point useHelixChat({ serverStop: { url } }) at it.

  • The route. A Cloudflare DO host calls the chat handler factory's stop({ sessionId, kind, reason }). Any other host calls handleChatStop (Express: through expressRequestToWebRequest).
  • Auth. A stop is a write: run the same auth and session-ownership check as the send route (handleChatStop's authorize is required).
  • Kinds. Accept only interrupt unless the app means to let a client destroy a turn (allowedKinds).
  • Only the explicit stop() posts. Unmounts, page refreshes and resync re-anchors stay local, so a refresh still resumes the run.
  • Target. The stop targets the session's current live run.

See the useHelixChat and handleChatStop references.

Resuming an Agent ​

Use handle.resume() to continue execution:

typescript
// Check if we can resume
const { canResume, reason } = await handle.canResume();

if (canResume) {
  // Resume execution
  const newHandle = await handle.resume();

  // Stream events from the resumed execution
  for await (const chunk of (await newHandle.stream()) ?? []) {
    console.log(chunk.type);
  }

  // Get final result
  const result = await newHandle.result();
  console.log(result.output);
}

Resume Modes ​

The resume() method supports four modes:

continue (default) ​

Resume from where the agent stopped:

typescript
const newHandle = await handle.resume(); // Same as { mode: 'continue' }
const newHandle = await handle.resume({ mode: 'continue' });

continue and with_message reject a session whose run is still executing, on every runtime: an active session with no pending client-tool or approval calls gets AgentAlreadyRunningError, and nothing is written. Before, Temporal and Cloudflare Workflows started a second run on it. An active session with pending calls is suspended and resumes normally. On Cloudflare Workflows handle.resume(options) runs executor.resume(), so it has the same guard and the same run-start commit before any cleanup or truncate (CP-83, CP-35), and it reactivates a stream the interrupt ended (CP-67). See the conformance findings.

with_message ​

Resume and add a new user message to the conversation:

typescript
const newHandle = await handle.resume({
  mode: 'with_message',
  message: 'Actually, focus on quantum entanglement specifically',
});

The message field accepts a string or an array of UserInputMessage objects for multi-message input:

typescript
const newHandle = await handle.resume({
  mode: 'with_message',
  message: [
    {
      role: 'user',
      content: 'Here is additional context from the system',
      metadata: { source: 'system' },
    },
    { role: 'user', content: 'Focus on quantum entanglement specifically' },
  ],
});

This is useful when the user wants to redirect the agent's focus. The messages are the new run's input rows: they are written in its run-start commit, so they are never lost to an interrupt before the first step commits.

with_confirmation ​

Resume a paused tool call with confirmation data:

typescript
// Agent paused waiting for confirmation on a tool
const newHandle = await handle.resume({
  mode: 'with_confirmation',
  data: { approved: true, notes: 'Proceed with the action' },
});

This mode is used when a tool requires human-in-the-loop approval. runtime-js and Cloudflare Workflows executor.resume() throw Resume mode "with_confirmation" is not yet implemented; submit the approval with submitToolResult and resume with continue instead (see Approval gates). The Cloudflare Durable Object's /resume body does not offer it at all (it accepts continue, with_message and from_checkpoint only; anything else is a 400 validation_error).

from_checkpoint ​

Resume from a specific historical checkpoint (time-travel):

typescript
// Get available checkpoints
const state = await handle.getState();
const checkpoints = await stateStore.listCheckpoints(handle.sessionId);

// Resume from an earlier checkpoint
const newHandle = await handle.resume({
  mode: 'from_checkpoint',
  checkpointId: checkpoints.checkpoints[0].id,
});

from_checkpoint rewinds both what the model sees and the durable log, on every runtime:

  • The message log is truncated to the checkpoint's messageCount, and the checkpoint's state (customState, stepCount) is restored on this session. The next model call sees exactly the history up to the checkpoint, plus any new input.
  • The checkpoint must belong to this session. An unknown id throws state_checkpoint_not_found; a checkpoint from another session throws state_checkpoint_foreign. Nothing changes in either case.
  • Pending client-tool or approval calls made after the checkpoint are discarded. Each gets a tool_end chunk with errorCode: 'aborted' on the stream, so your UI can settle it, and a later submitToolResult for one returns { status: 'unknown_tool_call' }. A client tool that was pending at the checkpoint and answered afterwards is pending again after the rewind if the store recorded it on the checkpoint; otherwise the next continuation closes it with a not-executed result, so the transcript stays valid.
  • run_resumed.fromCheckpointId and the onAgentResumed hook's resumedFromCheckpointId report the target checkpoint.
  • A session suspended on a client tool can be rewound too. On DBOS, the suspended workflow is cancelled and a fresh one starts.
  • Concurrent rewinds: at most one rewind and one run, on every runtime. The truncation rides the new run's run-start commit (StartRun.truncateTo), so the rewind and the run it starts land together or not at all. That includes a rewind to the checkpoint the session is suspended at, where the pending call is kept and re-stamped to the new run inside the same commit.
    • Every other concurrent resume() (a rewind, a continue or a with_message) loses the run-start commit, gets AgentAlreadyRunningError and writes nothing (on DBOS, it may already have cancelled the suspended workflow, which the winner cancels too).
    • A failure before the commit changes nothing. If the entry fails after its commit (for example its stream claim fails), the new run and the session end failed with a typed errorDetail, and the stream is failed (RunEntryFailedError).
  • A session whose run is still executing (active with no pending client-tool or approval calls, for example blocked in a long model call) cannot be rewound: from_checkpoint throws AgentAlreadyRunningError and changes nothing. Interrupt the run first, or wait for it to finish or suspend.

A completed or failed session is rewound too, on every runtime (time travel): the log is truncated to the checkpoint, the old output / error cleared, and a new run starts from there. (Before v0.50, Cloudflare Workflows and DBOS threw and Temporal returned the old terminal result; see the migration guide.) On the Cloudflare Durable Object host, the /resume body { agentType, options: { mode: 'from_checkpoint', checkpointId } } is honoured, as are continue and with_message (CP-34).

Resuming after HITL suspension (v7) ​

When a run reaches a client-tool boundary or an approval gate, the run loop suspends durably and handle.result() resolves with one of the suspended_* statuses. To continue:

typescript
const result = await handle.result();

switch (result.status) {
  case 'suspended_client_tool':
    // The client tool / approval gate is now pending submission.
    // Submit results via executor.submitToolResult({ kind, ... })
    // (or via chat.addToolOutput / chat.addToolApprovalResponse on the
    // AI SDK side), then resume:
    const resumed = await executor.resume({ sessionId: handle.sessionId });
    await resumed.result();
    break;

  case 'suspended_awaiting_children':
    // Parent is waiting for sub-agent(s) to finish their own HITL
    // cycles. Once each child resumes and completes, the framework
    // cascades the resume up to the parent automatically.
    break;

  case 'suspended_step_partial':
    // Mid-step suspend — some tools in the step ran, at least one is
    // pending submission. Submit + resume same as suspended_client_tool.
    break;

  case 'completed':
    /* handle output */ break;
  case 'failed':
    /* handle error */ break;
  case 'interrupted':
    /* user-requested pause */ break;
}

For chat-driven UIs, handleChatStream (the canonical chat handler in @helix-agents/ai-sdk) drives this lifecycle for you — submission + resume happens automatically on the server side, and AI SDK v6's stream-close-and-reopen lifecycle reattaches the client.

A partly answered batch is not resumable ​

When one step suspends on several client-tool or approval calls, resume() in continue or with_message mode waits for the whole batch. While any call of the batch still has no submitted result (and its deadline has not lapsed), every runtime refuses the resume with a retryable HelixError state_not_resumable and writes nothing. On runtime-js and the Cloudflare DO it is a ResumeRefusedError (sessionStatus: 'paused', awaitingToolCallIds) whose message names the awaited calls first, so they survive a typed HTTP body's 256-character scrub: Awaiting client tool call(s) <ids>: submit every result of the batch, then resume session '<id>'. The DO answers it 409 { error, code: 'state_not_resumable' }. Temporal, DBOS and Cloudflare Workflows throw a plain HelixError with the older wording (Cannot continue session '<id>': still awaiting results for client tool call(s) <ids>. …) and no awaitingToolCallIds. This is a behaviour change: DBOS used to attach to or continue the run, and a with_message resume could silently drop the message.

Use one of these instead:

  • Submit every result of the batch. The run continues once the last one lands (on the Cloudflare DO and DBOS automatically; elsewhere, call resume() then).
  • Stop the batch: call interrupt(), which ends a paused run at once. A continue resume of a run whose stop is recorded but not yet resolved (for example, recorded by another process) is not refused either: it ends the run by that stop (interrupted, or failed with framework_cancelled for an abort) with no model call. resume() reports that outcome on runtime-js, Cloudflare Workflows, Temporal and DBOS (on DBOS the parked workflow is cancelled first; on Cloudflare Workflows before any owed __submit__ continuation is adopted, so none is created); the Cloudflare DO's /resume answers 409 state_not_resumable ("the suspended run was stopped") and settles the run (onRunSettled once). A with_message resume over such a run settles it the same way and then runs as a new turn (Ruling 57), on every runtime; after an ABORT the settled session is failed, so the with_message is refused state_not_resumable (retry() is the path). A new turn (execute()) over such a run settles it the same way and then runs (Ruling 57a), after an abort too, since a failed session takes its next turn through execute(). This holds on every runtime, standard DBOS sessions included (DBOS persistent mode: FU-CP244-DBOS-PERSISTENT-MODE). Without a recorded stop, execute() over a session paused on client calls is still refused (AgentAlreadyRunningError).
  • Answer the rest with errors: submit an error result for each call you want to abandon, then resume.

A from_checkpoint resume is not refused. When the checkpoint keeps the unanswered calls, the new run takes them over and suspends again at once; when it discards them, it continues. See Client-executed tools for the submit side.

Crash Recovery ​

If your process crashes while an agent is running, the state is preserved in the state store. On restart, you can resume:

typescript
// After restart, reconnect to the session
const handle = await executor.getHandle(agent, savedSessionId);

if (handle) {
  const { canResume, reason } = await handle.canResume();

  if (canResume) {
    // Resume from last checkpoint
    const resumed = await handle.resume();
    const result = await resumed.result();
  }
}

For this to work, use a persistent state store like Redis:

typescript
import { RedisStateStore, RedisStreamManager } from '@helix-agents/store-redis';

const stateStore = new RedisStateStore({ host: 'localhost' });
const streamManager = new RedisStreamManager({ host: 'localhost' });
const executor = new JSAgentExecutor(stateStore, streamManager, llmAdapter);

What Happens During Resume ​

When you call resume() on a crashed or interrupted agent, the framework performs cleanup before continuing:

  1. Load checkpoint - Get the latest checkpoint (for from_checkpoint, the target checkpoint — see from_checkpoint)
  2. Run-start commit - One atomic commit truncates the messages past the checkpoint (for from_checkpoint always; for a continue on runtime-js and the Cloudflare DO, any rows past the latest checkpoint), appends a with_message input, supersedes the previous run and creates the new one. A caller that loses it changes nothing.
  3. Make the stream live - The winner claims the session stream, reactivates it and writes run_started
  4. Clean up orphaned staging - Remove any uncommitted staged changes
  5. Clean up stream chunks (runtime-js, both Cloudflare runtimes and DBOS; Temporal does not clean stream chunks) - Remove the previous run's chunks after the checkpoint step
  6. Emit stream_resync - Notify connected clients of the discontinuity: always after a from_checkpoint rewind (rollback) and a retry() (retry), on every runtime; after a plain resume only when step 5 removed chunks (crash_recovery)
  7. Continue execution - Resume from the checkpoint step
mermaid
graph LR
    subgraph Before ["Before Crash"]
        direction LR
        S1["Step 1"] --> S2["Step 2"] --> S3["Step 3"] --> S4["Step 4 ✗"]
    end

    S3 -. "Checkpoint" .-> S3

    subgraph After ["After Resume"]
        direction LR
        R1["Step 1"] --> R2["Step 2"] --> R3["Step 3"] --> R4["Step 4"] --> R5["Step 5"]
    end

    R3 -. "Resume from<br/>checkpoint" .-> R3

A step's messages are written by its own step commit, together with its checkpoint, so a crash in step 4 leaves no rows of step 4 behind on any runtime. The truncation is a guard for rows past the latest checkpoint, which only a pre-upgrade session can still hold; it removes them so they are not duplicated when the step re-executes.

Message Count Protection

The framework only truncates messages when a checkpoint exists AND has messageCount > 0. This prevents data loss in edge cases like:

  • First-step crash (no checkpoint yet)
  • Old checkpoints from before messageCount field was added

A run that finished just before the crash ​

If a process dies after a run's terminal commit landed but before its run record was marked terminal, the session already reads completed / failed / interrupted while the run still reads as a live run (running or suspended_*). Recovery does not re-run it: once its owner is gone, the run is ended with the session's committed status and its stream ended or failed (core finishCommittedRun; the Cloudflare DO wake and CF Workflows' readers reach it through core's closed-owner reconciliation, planClosedOwnerReconciliation → reconcileClosedOwnerRun). At an entry, StartRun.finishCurrent ends it in the run-start commit, whether it reads running or suspended_*, even while the old owner is still alive and its lease is valid (it has nothing left to write but its tail; its own later status write is a no-op). So a stop on a paused run whose K5 (interrupted) landed just before the crash is recorded interrupted by the next turn, never superseded.

Crash recovery of a still-running run is different from a resume. A resume() is an entry: it starts a new run with its own run-start commit and supersedes the stranded one once its owner is gone. The Cloudflare DO's wake path instead calls runtime-js's internal executor.recover(agent, sessionId, { runId }), which writes no run-start commit and re-enters the loop under the same runId; it refuses with RunSupersededError unless that run is still the session's current running run, and it finishes the run instead when its terminal commit already landed.

Resumable Statuses ​

An agent can be resumed when its status is:

StatusResumableDescription
runningYes¹Crash recovery — process died but state says running.
interruptedYesUser interrupted execution.
pausedYesWaiting for tool confirmation (legacy v6 path).
suspended_client_tool (v7)YesOne or more client-tool / approval calls are pending submission.
suspended_awaiting_children (v7)YesOne or more sub-agents are awaiting submissions; parent waits for cascade.
suspended_step_partial (v7)YesA step finished some tools but at least one is suspended; mid-step retry.
completedNo*Agent finished successfully.
failedNo*Agent failed with error.

¹ Only once the run's owner is gone (on JS, its lease has lapsed; on Temporal and DBOS, its workflow is closed). While the owner is live, resume() throws AgentAlreadyRunningError.

*resume() does not apply to terminal states — but a completed run is not a dead end. You can continue it as a new turn with execute({ sessionId }) (see Continuing a completed agent below), or time-travel into either terminal state with resume({ mode: 'from_checkpoint' }) on every runtime (see from_checkpoint).

The three new suspended_* statuses carry a result.suspended payload with the routing info (toolCallIds, children, stepId) the chat handler uses to drive resume. Most consumers do not read the payload directly — handleChatStream (from @helix-agents/ai-sdk) does it on their behalf — but it is part of the public type surface and exhaustive switches need to handle the variants.

Continuing a completed agent ​

resume() is for suspended runs (interrupted, paused, or a suspended_* HITL boundary). A completed run is continued differently: call execute({ sessionId }) against the completed session to start a new turn — it preserves the session's memory and produces a fresh result, with a fresh per-turn maxSteps budget.

typescript
// First turn runs to completion
const h1 = await executor.execute(agent, 'Draft a release note', { sessionId: 'rel-1' });
await h1.result(); // status: 'completed'

// Continue the SAME session as a new turn — memory is retained.
const h2 = await executor.execute(agent, 'Tighten the wording', { sessionId: 'rel-1' });
const r2 = await h2.result(); // fresh typed result, full prior history in context

A completed status can arrive before the previous run lets go (CP-84)

A run publishes completed in its terminal commit, then marks its run record terminal and ends its stream. Until the run record is terminal the finished run still owns the session, so an execute() for the next turn in that gap loses the run-start commit and is refused with AgentAlreadyRunningError. On runtime-js the refusal holds while the finished run's lease is valid; on Temporal, DBOS and Cloudflare Workflows while its workflow or instance is open. Through handleChatStream a refused send answers a retryable 409 { code: 'state_already_running' }. Retry after a short wait, or await result() on the previous handle, which resolves past the gap. See CP-84 in the conformance findings.

Fixed: the previous run's stream end no longer cuts the new turn (CP-85)

endStream, failStream and pauseStream are fenced on the stream's owner run. A finished run's late stream end is dropped once a newer run has claimed the stream, so it can no longer end the next turn's stream. Before, on JS and DBOS, the new turn failed with Cannot write to stream … in 'ended' state or streamed none of its output.

This now works for structured-output agents too. Previously a completed structured-output agent could not be continued on a real LLM provider: its transcript ended with a dangling __finish__ tool_use that had no matching tool_result, which Anthropic/OpenAI reject. The finish tool call is now paired with a synthetic { acknowledged: true } tool result, so the history is valid for continuation. See Finishing Agents — Continuing a completed structured-output agent.

The cross-runtime contract that execute(sessionId) is valid from completed is documented in the agent lifecycle table — see also the Agent Lifecycle Methods table in Core Concepts. The same continuation mechanism powers persistent companions: re-consulting a completed critic continues it on its preserved session. See the critic-loop recipe.

Stream Events ​

The framework emits stream chunks for interrupt/resume events:

run_interrupted ​

Emitted when an agent is interrupted:

typescript
{
  type: 'run_interrupted',
  runId: 'run-123',
  checkpointId: 'cpv1-run-123-s5-t1234567890-abc123',
  reason: 'user_requested'
}

run_resumed ​

Emitted when an agent resumes:

typescript
{
  type: 'run_resumed',
  runId: 'run-123',
  fromCheckpointId: 'cpv1-run-123-s5-t1234567890-abc123',
  fromStepCount: 5,
  mode: 'continue'
}

run_paused ​

Emitted when an agent pauses for confirmation:

typescript
{
  type: 'run_paused',
  runId: 'run-123',
  reason: 'tool_confirmation_required',
  pendingToolName: 'delete_file',
  pendingToolCallId: 'call-456'
}

checkpoint_created ​

Defined in the stream schema, but not written by any runtime today. Read checkpoints with stateStore.getLatestCheckpoint(sessionId) / listCheckpoints(sessionId). The chunk shape is:

typescript
{
  type: 'checkpoint_created',
  runId: 'run-123',
  checkpointId: 'cpv1-run-123-s5-t1234567890-abc123',
  stepCount: 5
}

Error Handling ​

AgentAlreadyRunningError ​

resume() throws this in every mode (continue, with_message and from_checkpoint), on every runtime, in two cases:

  • The run is still executing: the session is active with no pending client-tool or approval calls, or its current run still has a live owner (see One run at a time).
  • A concurrent entry won: this call lost the run-start commit to it (RunStartConflictError, whose name is AgentAlreadyRunningError).

Nothing is written in either case. An active session that does have pending calls is suspended, not running, and resumes normally (once its whole batch is answered, see A partly answered batch is not resumable).

typescript
import { AgentAlreadyRunningError } from '@helix-agents/core';

try {
  await handle.resume();
} catch (error) {
  if (error instanceof AgentAlreadyRunningError) {
    console.log(`Agent session ${error.sessionId} is already running`);
  }
}

AgentNotResumableError ​

Thrown when trying to resume an agent in a terminal state. It is a typed HelixError (code: 'state_not_resumable', never retryable; an optional hint is appended to the message):

typescript
import { AgentNotResumableError } from '@helix-agents/core';

try {
  await handle.resume();
} catch (error) {
  if (error instanceof AgentNotResumableError) {
    console.log(`Cannot resume: ${error.currentStatus}`);
  }
}

Terminal-session refusals differ per runtime (DI-31)

A running session is refused with AgentAlreadyRunningError in every mode. A completed or failed session (one an abort stop ended included) is refused:

  • on Cloudflare Workflows and DBOS, in continue or with_message mode, with AgentNotResumableError (code: 'state_not_resumable', non-retryable; the hint says to send the message with execute() as the next turn, or to retry() a failed session);
  • on Temporal, in with_message mode, with AgentNotResumableError too; a continue there returns the previous result instead (CP-60);
  • on runtime-js and the Cloudflare DO, in continue or with_message mode, with a typed ResumeRefusedError (code: 'state_not_resumable', sessionStatus, non-retryable).

Match them on HelixError.isInstance(error) && error.code === 'state_not_resumable', which covers both classes. See DI-31 in the conformance findings.

Completion-support refusal ​

An outputSchema agent whose model has no completion strategy is refused at every entry, resume() and retry() included, before its run-start commit (on runtime-js before any store read or write): framework_completion_strategy_unavailable (for example an Anthropic model id the adapter's capability table does not know) or framework_completion_schema_unsupported (a completion schema that native structured output cannot carry). Both are non-retryable HelixErrors; the session is left exactly as it was, so the same resume() / retry() succeeds once the adapter declares the model (the capabilities override). On the Cloudflare DO the HTTP routes answer 400 { error, code }. On Temporal the refusal comes from the worker's startRunCommit activity, before the run-start commit, and the executor rethrows it typed. DBOS does not refuse yet.

A pending forced-completion phase survives resume(), retry() and branching unchanged, and its next forced call resolves the strategy again against the adapter it now runs on: the strategy is never stored, so a session started on one model or adapter version can be resumed on another.

Runtime Considerations ​

JavaScript Runtime ​

The JS runtime handles a stop through its run registry. When you call interrupt():

  1. The stop is recorded on the run, and the run's controller is aborted with it
  2. At its next boundary (pollStop) the run ends: the in-flight step is not committed (its staged changes are discarded)
  3. The session is committed interrupted (the K5 terminal commit)
  4. The run loop exits cleanly

A run another process executes (a live foreign lease) gets the record only and ends at its next boundary; a run with no live owner is ended by the stop call itself.

Temporal Runtime ​

Temporal records the stop and wakes the workflow with a signal:

  1. interrupt() records the stop on the run, then signals the workflow (INTERRUPT_SIGNAL_NAME, carrying the StopRequest)
  2. The workflow reads the run's stop record (readStopActivity) at its next boundary and ends the run
  3. State is persisted via Temporal's durability guarantees
  4. Resume creates a new workflow execution (…__resume-<runId>, one per call) that continues from the checkpoint

A resume's run-start entry makes the session stream live again (claim, reactivate, run_started) before the resumed run writes, so the stream an interrupt ended accepts the resumed run's chunks. Earlier releases did not reactivate it (CP-76; see the Temporal runtime page).

Temporal: a failed resume() result can arrive before the failure is stored (CP-80)

The handle executor.resume() returns settles result() on the first error chunk, which the LLM activity writes before the workflow persists the failed status. When a resumed run (for example after a client-tool submission) ends on a terminal LLM error, result() can report failed while the stored session still reads active, and a retry() issued straight away throws Cannot retry: agent status is 'running'. Wait until the session reads failed before calling retry(). This was observed live with no injected delay, and the store caught up within about 1.5 s. A completed resume and the execute() / retry() handles are not affected. See CP-80 in the conformance findings.

Cloudflare Runtime ​

Cloudflare Workflows handle interrupts via instance coordination:

  1. interrupt() records the stop on the run and wakes the open instance with an event
  2. The workflow reads the run's stop record in a step.do at its next boundary and ends the run
  3. Resume creates a new workflow instance (…__resume__<key>, a key minted by that call) with a new run id, ${instanceId}:${agentType}-run-start:0. Instance ids are write-once, so every execute(), resume() and retry() runs in its own instance; getHandle() finds the live one from the session's current run record.

Interrupt Behavior During Sub-Agent Execution ​

When an agent spawns sub-agents (child workflows), interrupt signals propagate through the entire agent hierarchy. This enables responsive cancellation even during complex multi-agent operations.

How Interrupt Propagation Works ​

When you interrupt a parent agent that has running sub-agents:

  1. Parent receives the stop - The stop is recorded on the parent's run and its owner is signalled
  2. Children are stopped - The parent's stop reaches its running children (per runtime, below)
  3. Children stop gracefully - Each child ends at its next safe point
  4. Parent ends - The parent ends interrupted (or failed for an abort)

Target Response Time: < 1 second from interrupt request to stopped execution, even with deeply nested sub-agents.

Per-Runtime Implementation ​

RuntimeMechanismLatency
JSThe stop record + the run's aborted signal; in-process children are stopped with itImmediate (< 100ms)
TemporalThe stop record (readStopActivity) + the INTERRUPT_SIGNAL_NAME wake-up< 1 step boundary
Cloudflare DOThe stop record + the drive's aborted signal; children stopped through their routes¹< 1 step boundary
Cloudflare WorkflowsThe stop record (read in a step.do, and polled while a step runs) + an event wake-up²Mid-step (~500 ms)
DBOSThe stop record (readStopStep) + DBOS.cancelWorkflow< 1 step boundary

² An in-flight model call, tool or hook gets an abortSignal that fires within stopPollIntervalMs (default 500 ms) of the stop, and a run parked on a sub-agent or a blocking companion wakes on the stop and propagates it to that child.

¹ The DO's /interrupt (and EndpointContext.stop) records the stop, signals the executing drive and answers at once (202); it never waits for the run. A run that ignores its abortSignal is ended by the stop deadline (stopDeadlineMs, default 30 s). A new /start, /resume or /retry while a run executes is refused 409 state_already_running by default; set concurrentEntry: 'interrupt_previous' to have the new entry stop the live run (after its own gates and beforeEntry pass) and wait for it to settle, bounded by the stop's deadline plus 5 s. This replaces calling executionState.interrupt() from a beforeStart hook. A chat send (createCloudflareChatHandler) is refused 409 while a run executes even under interrupt_previous (CP-13), until chat sends carry a request id: stop the run first, then send. See the DO drive pipeline upgrade guide.

JavaScript Runtime ​

The JS runtime uses AbortSignal for cooperative cancellation:

typescript
// Parent's AbortController is linked to child agents
// When parent.abort() is called, children receive abort signal immediately

// In custom tools, check the signal:
const myTool = defineTool({
  execute: async (input, context) => {
    for (const item of items) {
      if (context.abortSignal.aborted) {
        return { partial: true, processed: results };
      }
      // ... process item
    }
  },
});

Temporal Runtime ​

handle.interrupt() records the stop on the run and signals its workflow (INTERRUPT_SIGNAL_NAME); a stop recorded from any process works the same way. The workflow reads the run's stop record before each step's LLM call and at a resume's entry, never while a step's LLM call, tools or sub-agent children are in flight, so they finish first. Child workflows are not signaled: a sub-agent child runs to its own end, and a non-blocking persistent companion keeps running. See Temporal → Interrupts and sub-agents.

Cloudflare Runtime ​

Cloudflare Workflows records the stop on the run and wakes the instance with an event:

  1. The stop is recorded on the run (requestStop)
  2. An interrupt event is sent to the open instance (instance.sendEvent()) for an immediate wake-up
  3. The workflow reads the run's stop record in a step.do before each step and during a sub-agent wait
  4. On a stop during a sub-agent wait, the stop is recorded on each pending child's run, and the parent ends by it

A paused run, or one whose instance already closed, is ended by the stop call itself.

Best Practices for Sub-Agent Interrupts ​

  1. Keep sub-agent operations bounded - Long-running sub-agents delay interrupt response
  2. Use interruptible tools - Check context.abortSignal.aborted in loops
  3. Configure poll intervals appropriately - Shorter intervals = faster response, more overhead
  4. Design for partial results - Sub-agents should return meaningful partial output when interrupted

Best Practices ​

1. Always Check canResume() ​

Before resuming, verify the agent can be resumed. canResume() is only a read: a concurrent caller can still win the run start, so handle AgentAlreadyRunningError as well (below).

typescript
const { canResume, reason } = await handle.canResume();
if (!canResume) {
  console.log(`Cannot resume: ${reason}`);
  return;
}

2. Handle Resume Errors ​

Wrap resume calls in try-catch:

typescript
try {
  const resumed = await handle.resume();
  await resumed.result();
} catch (error) {
  if (error instanceof AgentAlreadyRunningError) {
    // Another process resumed first
  } else if (error instanceof AgentNotResumableError) {
    // Agent is in terminal state
  }
}

A completed or failed session is refused with AgentNotResumableError only on Cloudflare Workflows; runtime-js and the Cloudflare DO throw ResumeRefusedError (also state_not_resumable), and DBOS a plain Error (DI-31; see Error Handling), so match on the code and also handle an untyped error.

3. Use Persistent Storage for Production ​

For crash recovery to work, use Redis or another persistent store:

typescript
const stateStore = new RedisStateStore({
  host: process.env.REDIS_HOST,
  ttl: 86400 * 7, // 7 days retention
});

4. Store Session IDs for Later Resumption ​

Save session IDs so you can reconnect after crashes:

typescript
const handle = await executor.execute(agent, input);

// Persist the session ID
await database.save({ sessionId: handle.sessionId, userId });

// Later, after restart
const savedSession = await database.get(userId);
const handle = await executor.getHandle(agent, savedSession.sessionId);

Next Steps ​

Released under the MIT License.