Skip to content

Temporal Runtime ​

The Temporal runtime (@helix-agents/runtime-temporal) executes agents as durable Temporal workflows. This provides crash recovery, automatic retries, and production-grade reliability for long-running agent tasks.

When to Use ​

Good fit:

  • Production workloads requiring reliability
  • Long-running agents (hours or days)
  • Agents that must survive process restarts
  • Complex multi-agent orchestrations
  • Operations requiring audit trails and observability

Not ideal for:

  • Quick development iteration (infrastructure overhead)
  • Simple, short-lived agents
  • Cost-sensitive deployments without existing Temporal infrastructure

Prerequisites ​

You need a running Temporal server:

Option 1: Temporal Cloud (Recommended for production)

bash
# Sign up at https://temporal.io/cloud

Option 2: Local development

bash
# Using Docker
docker run -d --name temporal \
  -p 7233:7233 -p 8233:8233 \
  temporalio/auto-setup:latest

# Or using Temporal CLI
temporal server start-dev

Installation ​

bash
npm install @helix-agents/runtime-temporal @helix-agents/store-redis @temporalio/client @temporalio/worker @temporalio/workflow @temporalio/activity zod

The @temporalio/* peer dependencies are ^1.13.0 (they were ^1.11.0), and zod (^4.1.8) is a peer dependency too.

Architecture ​

mermaid
graph TB
    subgraph App ["Your Application"]
        Executor["<b>TemporalAgentExecutor</b><br/>Starts workflows · Returns handles<br/>Reads from StateStore/StreamManager"]
    end

    subgraph Server ["Temporal Server"]
        ServerDesc["Persists workflow state<br/>Manages task queues<br/>Handles retries and timeouts"]
    end

    subgraph Worker ["Temporal Worker"]
        subgraph Workflow ["Agent Workflow"]
            WF["Orchestrates execution<br/>Calls activities for LLM/tools"]
        end
        subgraph Activities ["Activities"]
            Act["LLM calls · Tool execution<br/>State persistence"]
        end
    end

    App --> Server --> Worker

Setup Guide ​

1. Create the Workflow ​

Register the framework's agentWorkflow under the name your executor's workflowName expects:

typescript
// src/workflows.ts
import { agentWorkflow as runAgentWorkflow } from '@helix-agents/runtime-temporal/workflow';
import type { AgentWorkflowInput, AgentWorkflowResult } from '@helix-agents/runtime-temporal';

export async function agentWorkflow(input: AgentWorkflowInput): Promise<AgentWorkflowResult> {
  return runAgentWorkflow(input);
}

The workflow proxies every GenericActivities method, runs the step loop, the run-start commit, the suspension and terminal commits, and child-workflow dispatch.

2. Create Activities ​

Activities perform the actual work (LLM calls, tool execution, state and stream writes). Configure one GenericActivities instance and register every method on its prototype:

typescript
// src/activities.ts
import { AgentRegistry, GenericActivities } from '@helix-agents/runtime-temporal';
import { VercelAIAdapter } from '@helix-agents/llm-vercel';
import { RedisStateStore, RedisStreamManager } from '@helix-agents/store-redis';

const registry = new AgentRegistry();
registry.register(ResearchAgent);
registry.register(AnalyzerAgent);

export function createActivities(): Record<string, (...args: never[]) => unknown> {
  const activities = new GenericActivities({
    stateStore: new RedisStateStore(redisClient),
    streamManager: new RedisStreamManager(redisClient),
    llmAdapter: new VercelAIAdapter(),
    agentResolver: {
      resolveAgent: (type) => registry.resolve(type, { env: undefined, sessionId: '', runId: '' }),
    },
  });
  // Bind every prototype method: the workflow calls activities a hand-written
  // list would miss (`Activity function <name> is not registered on this Worker`).
  const proto: object = Object.getPrototypeOf(activities);
  return Object.fromEntries(
    Object.getOwnPropertyNames(proto)
      .filter((name) => name !== 'constructor')
      .map((name) => [name, Object.getOwnPropertyDescriptor(proto, name)?.value] as const)
      .filter(
        (e): e is readonly [string, (...args: never[]) => unknown] => typeof e[1] === 'function'
      )
      .map(([name, fn]) => [name, fn.bind(activities)] as const)
  );
}

This release adds activities (the run-start, suspension and terminal commits, among others), so a worker that registers a hand-written list of activity names must be updated.

Optional GenericActivities dependencies for ownership and attempts:

OptionDefaultPurpose
ownerProbedescribe() via the activity contextAnswers whether a live run's owner execution (${workflowId}:${workflowRunId}) is still running. Tests inject one
ownerWait30 s, polling from 100 msBounds an entry's wait for a suspending owner workflow to close ({ timeoutMs, initialIntervalMs })
workflowProbedescribe() via the activity contextDescribes a workflow by id; used to recognise a stalled companion continuation
llmAttemptFenceIntervalMs2 000How often a running runLLMStep attempt heartbeats and checks for a newer attempt of its step (a superseded attempt aborts)

The default probes use the activity context's client, which needs @temporalio/activity 1.13 or later.

3. Create the Worker ​

The worker processes workflows and activities:

typescript
// src/worker.ts
import { Worker, NativeConnection } from '@temporalio/worker';
import { createActivities } from './activities';

async function runWorker() {
  const connection = await NativeConnection.connect({
    address: process.env.TEMPORAL_ADDRESS ?? 'localhost:7233',
  });

  const worker = await Worker.create({
    connection,
    namespace: 'default',
    taskQueue: 'agent-tasks',
    workflowsPath: require.resolve('./workflows'),
    activities: createActivities(),
  });

  await worker.run();
}

runWorker().catch(console.error);

4. Create the Executor ​

The executor starts workflows and returns handles:

typescript
// src/executor.ts
import { Client, Connection, WorkflowNotFoundError } from '@temporalio/client';
import { TemporalAgentExecutor } from '@helix-agents/runtime-temporal';
import { RedisStateStore, RedisStreamManager } from '@helix-agents/store-redis';

async function createExecutor() {
  const connection = await Connection.connect({
    address: process.env.TEMPORAL_ADDRESS ?? 'localhost:7233',
  });

  const client = new Client({ connection });

  // Wrap Temporal client to match interface
  const temporalClientAdapter = {
    startWorkflow: async (name, options) => {
      const handle = await client.workflow.start(name, {
        workflowId: options.workflowId,
        taskQueue: options.taskQueue,
        args: options.args,
      });
      return wrapHandle(handle);
    },
    // `runId` pins one execution of a reused id (a stop targets the run's own execution).
    getHandle: (workflowId, runId) => {
      return wrapHandle(client.workflow.getHandle(workflowId, runId));
    },
    // REQUIRED: the liveness of ONE workflow execution (see "Run start and ownership").
    describeWorkflow: async (workflowId, runId) => {
      try {
        const description = await client.workflow.getHandle(workflowId, runId).describe();
        return { status: description.status.name }; // 'RUNNING', 'COMPLETED', ...
      } catch (error) {
        if (error instanceof WorkflowNotFoundError) return null; // closed and past retention
        throw error;
      }
    },
  };

  return new TemporalAgentExecutor({
    client: temporalClientAdapter,
    stateStore: new RedisStateStore(redis),
    streamManager: new RedisStreamManager(redis),
    workflowName: 'agentWorkflow',
    taskQueue: 'agent-tasks',
    // entryOutcomeTimeoutMs: 120_000, // run-start fail-safe (default)
  });
}

Breaking: TemporalClient.describeWorkflow is required

TemporalClient gained a required describeWorkflow(workflowId, runId) method. It returns the execution's status name, or null when the server no longer knows the execution. The executor uses it to refuse an entry while the current run's workflow is still running, and to finish a run whose workflow closed between its terminal commit and its terminal run status. An adapter without it does not type-check. See the upgrade guide.

Using the Executor ​

Once set up, usage is identical to other runtimes:

typescript
const executor = await createExecutor();

// Execute agent with a string message
const handle = await executor.execute(ResearchAgent, 'Research quantum computing');

// Or with multiple messages (for context injection, file attachments, etc.)
// const handle = await executor.execute(ResearchAgent, {
//   message: [
//     { role: 'user', content: 'Focus on recent breakthroughs', metadata: { source: 'system' } },
//     { role: 'user', content: 'Research quantum computing' },
//   ],
// });

// Stream events
const stream = await handle.stream();
for await (const chunk of stream) {
  console.log(chunk);
}

// Get result
const result = await handle.result();

Conversation History ​

Fixed: Temporal sent the model no conversation history (RM-10)

Before this release, every Temporal LLM call received only the system prompt. The workflow projected session state through a messages field that SessionState does not have, so the model never saw the user's message, earlier turns or tool results. With a real provider every call failed with messages must not be empty, from the first turn on; with a mock adapter the runs appeared to work. Hook messageIndex values were computed from the same empty array. Upgrade @helix-agents/runtime-temporal and @helix-agents/core together.

runLLMStep now loads the complete, count-checked history inside the activity with core loadLLMHistory and builds the model input from it. History never travels through workflow code or activity payloads (Temporal payload size limits, and every payload is recorded in workflow history). The activity returns only what the step produced:

RunLLMStepResult fieldMeaning
nextState.messagesAlways []
newMessagesMessages this step appended (assistant message, synthetic results), in order
baseMessageCountLength of the persisted history the step was built from (base for absolute hook indices)

If the store returns fewer rows than its live count (after the loader's bounded re-reads), the activity fails with a non-retryable ApplicationFailure carrying the classified ErrorDetail, and the run ends failed with errorDetail.code === 'state_history_incomplete'. No model call is made and the activity is not retried. If a store call throws while loading (for example a dropped Redis or Postgres connection), the ApplicationFailure is retryable (errorDetail.retryable: true), so the activity retry policy applies: a transient blip is retried and the run proceeds. Only when every attempt fails does the run end failed with the same code.

Deploying this version: drain, and never mix old and new workers

The workflow code and the runLLMStep result shape changed together, with no patched() guard. New workflow code replaying an old result throws TypeError: Cannot read properties of undefined (reading 'length') on every workflow task and the workflow hangs in Running (no history is lost; terminate or reset it). Old workflow code receiving a new result persists nothing from the empty nextState.messages: the assistant message is silently lost and its tool results are orphaned.

  1. Drain in-flight Temporal workflows before deploying.
  2. Never run old and new workers on the same task queue: cut over fully, or use Worker Versioning or a new task queue.
  3. Terminate or reset any workflow stuck on the workflow-task error Cannot read properties of undefined (reading 'length').

Run Start and Ownership ​

Every entry (execute(), resume(), retry(), a child or companion turn) starts its run with one run-start commit, written by the workflow's entry activity: appendMessages(sessionId, inputRows, { streamSequence, runId, startRun }). It appends the input rows, applies any rewind (from_checkpoint, retry()), creates the run record and its input checkpoint, all atomically. Nothing is written before it: the only executor-side write is a new session's write-once createSession. Only the winner then claims the stream (a compare-and-set), makes it live and writes run_started; a loser writes nothing and its call throws AgentAlreadyRunningError (RunStartConflictError) or RunStartRejectedError.

  • Run ids. The executor mints the run id and passes it in the workflow input, so a replay reads the same value.
  • Owner token. A run's ownerToken is ${workflowId}:${workflowRunId}: one Temporal execution. An entry into a session whose current run is still live is refused while that execution is RUNNING (describeWorkflow); once it is closed (completed, failed, terminated, timed out) the run is ownerless and the next entry supersedes it, so a terminated or crashed workflow never bricks the session.
  • Fail-safe. execute() / resume() / retry() wait until their entry has an outcome (its run committed and wrote run_started, or it lost). If none arrives within entryOutcomeTimeoutMs (default 120 000 ms; for example no worker polls the task queue), the call gives the entry up atomically with the store (abandonRunStart, a tombstone that refuses a late commit), terminates the workflow and throws core RunStartAbandonedError (transport_error, retryable once the workflow was terminated). If the tombstone cannot be written, it throws RunStartOutcomeUnknownError (non-retryable: read the session before retrying). Both are exported by @helix-agents/core.
  • Hooks. A run that fails at its entry (the run-start commit, the stream claim or run_started) fires no lifecycle hook; onAgentFail fires only for a run whose onAgentStart / onAgentResumed fired. The start hooks are recorded durably (agent-start-hooks-<runId>): an activity retry replays their writes instead of re-firing them, but a worker crash in the middle of them re-fires them (at-least-once). A start hook that throws fails the entry non-retryably.
  • Terminal order. A terminal exit (completion, failure, stop) writes its terminal commit (commitState, a state checkpoint carrying the status, error and the terminal hooks' writes), then the run's terminal status, then ends or fails the stream. The finalization after the terminal commit runs on a long retry budget, terminalFinalizeTimeoutMs in the workflow input (default 3 minutes). Past it the workflow closes with the committed outcome, and the next getHandle() / resume() finishes the run.
  • Suspension order. The step's frames (tool_start / tool_approval_request), then the suspension commit with the run's suspended_* status, then pauseStream (run_paused). The stream's paused status means the suspension is committed.
  • Superseded runs. Every write carries the run id and is fenced to it; a run that was taken over stops with AgentResult.status === 'superseded' (errorDetail.code === 'state_run_superseded', ApplicationFailure.nonRetryable). onAgentSuperseded fires, never onAgentFail, and onAgentComplete only when the supersede landed after it fired and before the terminal commit (then complete → superseded, the last hook authoritative).
  • Attempts. Every chunk of an LLM attempt carries runId and an attemptId, and step_start is the attempt's first chunk. An activity retry re-streams the step under a new attemptId; readers drop the superseded attempt's content. An error chunk is written only for a terminal failure, never for an attempt that will be retried.
  • resumeCount is bumped by the winning resume() / retry() entry, after its run-start commit (not for a child re-spawn).

Deploying this release: drain or terminate in-flight workflows

This release changes the workflow's command order with no patched() guard: a suspension now runs commitSuspendedStep → the suspension marker → pauseSuspendedStream, a resume's first command is startRunCommit, commitTerminalState + finalizeTerminalRun replace persistTerminalState, activities were added, and workflow ids changed (__resume-<runId>, __retry__<runId>, a companion continuation …__continue__<toolCallId>). A workflow started by the previous release cannot replay on the new code (nor the reverse, so a rollback needs the same drain): it may get stuck retrying a nondeterministic workflow task, or be failed. Either way, before you deploy, let in-flight Temporal workflows finish or terminate them, and never run old and new workers on the same task queue. A terminated run's session is then superseded by its next entry. See the deploy checklist.

Multi-Turn Conversations ​

The Temporal runtime supports the same multi-turn conversation API as the JS runtime using the session-centric model:

Using sessionId ​

typescript
// First message - creates a new session
const handle1 = await executor.execute(agent, 'Hello, my name is Alice', {
  sessionId: 'session-123',
});
await handle1.result();

// Continue the conversation - same sessionId
const handle2 = await executor.execute(agent, 'What is my name?', {
  sessionId: 'session-123',
});

The next turn can be refused just after the previous one reads completed (CP-84)

The terminal path writes the terminal commit (the session reads completed), then the run status, then ends the stream, and the workflow closes only after that. Every execute() turn runs under the session's base workflow id (agent__<name>__<sessionId>), so an execute() for the next turn in that gap is refused with AgentAlreadyRunningError: by the owner check while the run's workflow is still RUNNING, or by the workflow-id collision ("session was claimed by a concurrent request"), although nothing else is running. The same call succeeds once the old workflow closes. Wait for handle1.result() (as above) before sending the next turn; a client that sends as soon as it reads completed should retry the refusal briefly. Observed live. See CP-84 in the conformance findings.

Using handle.send() ​

typescript
const handle1 = await executor.execute(agent, 'Hello', {
  sessionId: 'session-123',
});
await handle1.result();

const handle2 = await handle1.send('Tell me more');

Using Direct Messages ​

typescript
const handle = await executor.execute(agent, {
  message: 'Continue from here',
  messages: myExternalMessageHistory,
});

Multi-Message Input ​

The message field in all operations accepts either a string or a UserInputMessage[] array. Multi-message input lets you inject context alongside the user's question, attach files, or send multiple messages in a single execution:

typescript
const handle = await executor.execute(
  agent,
  {
    message: [
      {
        role: 'user',
        content: 'Background: user is on the enterprise plan',
        metadata: { source: 'system' },
      },
      { role: 'user', content: 'What features do I have access to?' },
    ],
  },
  { sessionId: 'session-123' }
);

// Messages can include file attachments
const handle = await executor.execute(
  agent,
  {
    message: [
      {
        role: 'user',
        content: 'Analyze this image',
        files: [
          { data: 'data:image/png;base64,...', mediaType: 'image/png', filename: 'chart.png' },
        ],
      },
    ],
  },
  { sessionId: 'session-123' }
);

Multi-message input also works with handle.send() and resume with_message mode:

typescript
const handle2 = await handle1.send([
  { role: 'user', content: 'Additional system context' },
  { role: 'user', content: 'Follow-up question' },
]);

const resumed = await handle.resume({
  mode: 'with_message',
  message: [{ role: 'user', content: 'Here is what I meant instead' }],
});

String and multi-message inputs can be mixed freely across turns in the same session.

Behavior Table ​

InputMessages SourceState Source
message only (new session)Empty (fresh)Empty (fresh)
message + sessionId (existing)From sessionFrom session
message + messagesFrom messagesEmpty (fresh)
message + stateEmpty (fresh)From state
message + sessionId + messagesFrom messages (override)From session
message + sessionId + stateFrom sessionFrom state (override)
All fourFrom messages (override)From state (override)

Note: In the behavior table, message can be either a string or UserInputMessage[]. Both forms work identically with all combinations.

Caller-provided history: prefix: The agent-input history: prefix is honored on fresh sessions — the executor pre-claim no longer drops it, so the seeded history lands in the message log and onMessage fires with the correct messageIndex. On continuation turns (an existing session with a non-empty durable message log) the prefix is dropped — this is now the uniform behavior across all runtimes (history: is fresh-session seeding only). The drop is observable via a temporal.history_dropped_on_continuation warn. The earlier runtime-cloudflare divergence (CF appended the prefix on continuations, doubling the durable log) was resolved by gating every runtime on an empty durable log, per FU-CROSS-RUNTIME-HISTORY-CONTINUATION-DOUBLING.

See JS Runtime - Multi-Turn Conversations for detailed documentation.

Execute vs Resume Semantics ​

The Temporal runtime provides two methods for running agents with different semantics:

MethodUse CaseStream BehaviorSession Behavior
execute()Start new conversation or add new messageCreates new stream segmentCreates or continues session
resume()Continue interrupted sessionContinues existing streamMust have interrupted state

execute() is for normal agent interactions:

  • Starts a new Temporal workflow (or continues an existing session)
  • Creates a fresh stream segment for the response
  • Appends the new user message to the session
typescript
// New conversation
const handle = await executor.execute(agent, 'Hello', { sessionId: 'session-1' });

// Follow-up in same session (new run, new stream segment)
const handle2 = await executor.execute(agent, 'Tell me more', { sessionId: 'session-1' });

resume() is specifically for recovering from interrupts or HITL suspensions:

  • Starts a NEW Temporal workflow instance with the __resume-<runId> suffix (the prior workflow has already exited at the interrupt or HITL boundary)
  • Maintains stream continuity for clients reconnecting (the new workflow emits a run_resumed chunk so the SSE client can dedupe)
  • Session must be in interrupted, paused, or HITL-suspended status (pendingClientToolCalls map non-empty)
typescript
// After an interrupt...
const state = await stateStore.loadState('session-1');
if (state?.status === 'interrupted') {
  // Resume starts a new workflow that continues the same stream
  const handle = await executor.resume(agent, 'session-1');
}

Resume conventions (post-A.2) ​

The Temporal runtime exits the workflow on every HITL boundary (client-executed tool, approval gate, sub-agent suspension). executor.resume(sessionId) starts a NEW workflow instance with a distinctive workflow ID:

${workflowIdPrefix}__${agentType}__${sessionId}__resume-${runId}
  • runId is the id the call minted for its run, so the id is unique per entry and the caller writes no counter before it knows whether it wins (it used to be __resume-<N> from incrementResumeCount()).
  • Single-dash separator (__resume-<runId>), distinct from retry()'s __retry__<runId>. A sub-agent re-spawned by its parent workflow still uses __resume-<n> (n from the child's resumeCount + 1), and a companion continuation …__persistent__<childSessionId>__continue__<toolCallId>.
  • getHandle() finds the workflow that owns the session's current run from the run's owner token, never by rebuilding an id.
  • ALLOW_DUPLICATE is still passed (harmless with a per-run id).

The new workflow's mode='resume' branch calls activities.applyResultsAndReload, which:

  1. Drains submitted client-tool results into the message log.
  2. Fires onMessage + afterTool hooks for each drained entry.
  3. Synthesizes client_tool_timeout results for expired deadlines.
  4. Drains completed sub-agent children via recordSubSessionResult.
  5. Returns { state, drainedCount, hasMoreSuspended, childrenToRespawn } so the workflow body can decide whether to continue or re-suspend.

A partial client-tool batch is not resumable (R44, CP-86). resume() in continue / with_message mode on a session whose client-tool batch is still partial (some awaited call has no submitted result) is refused with a retryable state_not_resumable before any write or workflow start. Submit every result of the batch first, or interrupt(), or submit error results for the calls you want to abandon. A from_checkpoint rewind is not refused: its new run re-suspends at once on any restored call that is still unanswered. When a resumed run re-suspends without a step (hasMoreSuspended, for example on children that are still suspended), it writes its suspended_* run status, fires onAgentSuspended and pauses the stream, so the session reads paused again (CP-86 and CP-65 are fixed by this release; see the conformance findings).

AgentResult.status for HITL agents resolves with one of:

  • 'suspended_client_tool' — at least one client-executed tool is awaiting submission.
  • 'suspended_awaiting_children' — at least one sub-agent child is in 'suspended_*' and the parent is awaiting it.
  • 'suspended_step_partial' — mixed phase-1 step where some tools completed and others suspended.

result.suspended carries the routing info ({ toolCallIds?, children?, stepId? }) needed to drive the next submitToolResult + resume cycle.

A suspension now pauses the root session's stream after the suspended run status and the internal suspension_marker, so a reader sees run_paused (the AI SDK's data-run-paused) as on JS (it used to stay active, RM-56). A child session writes into its parent's stream and never pauses it. The finding record stays open until its conformance cell lands; see RM-56 in the conformance findings.

abort() mid-step (WA MR2 F4-C). An abort() while a step is in flight cancels the workflow before the step commits. The workflow's terminal runs in a non-cancellable scope, so the cancelled workflow still commits the abort's own K5 (failed / framework_cancelled, resolved from the stop record). That terminal marks the uncommitted step step_discarded (Ruling 66) before it fails the stream. interrupt() is not affected: it commits the in-flight step first. Both are fixed in code; the CP-22 and RM-59 records in the conformance findings flip when the conformance owner re-runs their cells.

Resume modes. The mode rides the workflow input, and the resume workflow's entry performs it inside its run-start commit (previously every mode behaved like continue, CP-31):

  • from_checkpoint — the checkpoint is resolved before any write (another session's id throws state_checkpoint_foreign, an unknown id state_checkpoint_not_found, both before any workflow starts). The rewind is the run-start commit's truncateTo: the message log is truncated to the checkpoint's messageCount atomically with the new run, its state restored, and each discarded pending client-tool call gets an aborted tool_end. An active session whose run is still live-owned (no pending client-tool or approval calls) is refused with AgentAlreadyRunningError, and concurrent calls have one winner. A completed or failed session is rewound too and a new workflow starts from the checkpoint (C7, as on every runtime); only continue takes the terminal short-circuit, which returns the old terminal result. A with_message over a completed or failed session is refused with AgentNotResumableError (state_not_resumable) before any write, as on every other runtime: retry() a failed session, or send the next turn with execute(). That includes a session an abort stop ended (failed / framework_cancelled).
  • with_message — the message is the run-start commit's input rows, so the first model call sees it, and a call that loses writes nothing.

submitToolResult({ sessionId, toolCallId, result | error }) is a durable write only — no Temporal signal is sent. The workflow has already exited at the HITL boundary; consumers drive the next loop iteration via executor.resume() (or via the framework's handleChatStream plumbing on the server side).

A submit for a call of a stopped run stores nothing (WA MR2 §7.1): the submit reads the stop of the call's run on its owner session first, and the answer is { status: 'unknown_tool_call', reason: 'run_stopped' }. A stop that is recorded but not yet resolved is settled through the stop path (stopRun with the record's own kind and reason):

  • the run's workflow has exited (the usual case: a Temporal workflow exits at a HITL boundary), so the stop's terminal commits before the answer (interrupted, or failed / framework_cancelled);
  • the run's workflow is still open (for example, still in its suspension tail), so it is only signalled (INTERRUPT_SIGNAL_NAME; an abort also cancels it), never terminated. The answer is still run_stopped, but it arrives before the terminal lands: the workflow ends the run by the stop when it reads the record, and a session read right after the answer can still show the suspended run.

A stop that already ended the run answers the same while the call is still unanswered in the log.

Agent Registry ​

Register agents so the worker can instantiate them:

typescript
import { AgentRegistry } from '@helix-agents/runtime-temporal';

const registry = new AgentRegistry();

// Register each agent type
registry.register(ResearchAgent); // name: 'researcher'
registry.register(AnalyzerAgent); // name: 'analyzer'
registry.register(SummarizerAgent); // name: 'summarizer'

// The built-in activities resolve agents by type:
// new GenericActivities({
//   agentResolver: { resolveAgent: (type) => registry.get(type) }, // static agents only
//   stateStore, streamManager, llmAdapter,
// })

// In an activity of your own, look up by type
export async function myAgentActivity(agentType: string) {
  const agent = registry.get(agentType); // Returns the agent config
  // ...
}

AgentRegistry with Factories ​

The AgentRegistry supports both static registration and factory functions for dynamic agent creation. Factories are useful when agents need runtime dependencies like API keys, database connections, or per-request configuration.

Static vs Factory Registration ​

typescript
import { AgentRegistry } from '@helix-agents/runtime-temporal';

const registry = new AgentRegistry();

// Static registration - for agents with no runtime dependencies
registry.register(ResearchAgent);
registry.register(SummarizerAgent);

// Factory registration - for agents needing runtime dependencies
interface Env {
  TAVILY_API_KEY: string;
  DATABASE: Database;
}

registry.registerFactory<Env>('dynamic-researcher', (ctx) => {
  return createResearchAgent({
    apiKey: ctx.env.TAVILY_API_KEY,
    database: ctx.env.DATABASE,
    userId: ctx.userId,
    sessionId: ctx.sessionId,
  });
});

AgentFactoryContext ​

Factory functions receive context about the current execution:

typescript
interface AgentFactoryContext<TEnv = unknown> {
  /** Environment bindings (e.g., API keys, connections) */
  env: TEnv;
  /** Current session identifier */
  sessionId: string;
  /** Current run identifier */
  runId: string;
  /** Optional user identifier */
  userId?: string;
}

Using resolve() in Activities ​

The resolve() method handles both static and factory agents. Use it instead of get() when you have factories, for example in an activity of your own:

typescript
// src/activities.ts
export async function myAgentStepActivity(
  agentType: string,
  state: AgentState<unknown, unknown>,
  context: { sessionId: string; runId: string; userId?: string }
): Promise<StepResult> {
  // resolve() handles both static and factory agents
  const agent = registry.resolve(agentType, {
    env, // Your environment object
    sessionId: context.sessionId,
    runId: context.runId,
    userId: context.userId,
  });

  return executeStep(agent, state, llmAdapter, streamManager);
}

Error Handling for Unknown Agent Types ​

An unknown agent type throws core AgentResolutionError.notFound (code: 'framework_agent_not_found', non-retryable, with agentType and availableTypes) from both get() and resolve(). get() returns only static registrations: for a factory-registered type it throws the same code with a message that says to use resolve(). Detect a miss with core isAgentNotFoundError, which matches by name and code (not instanceof). The registry stays synchronous (resolve() returns the config, never a promise), because activities resolve through your synchronous resolveAgent.

typescript
import { isAgentNotFoundError } from '@helix-agents/core';

try {
  const agent = registry.resolve('unknown-agent', context);
} catch (error) {
  if (isAgentNotFoundError(error)) {
    console.error(`Unknown agent type: ${error.agentType}`);
    console.error(`Available types: ${error.availableTypes?.join(', ')}`);
  }
}

A miss that escapes an activity still reaches the run as the wrapping ActivityFailure's detail, after the activity retries, not as framework_agent_not_found (DI-04). See the DO drive pipeline upgrade guide.

Registry Methods ​

MethodDescription
register(config)Register a static agent configuration
registerFactory(type, factory)Register a factory function
replace(config)Overwrite an existing static registration
resolve(type, context)Resolve agent (factory-first, then static)
get(type)Get a static registration only
has(type)Check if agent type is registered
unregister(type)Remove an agent registration
getRegisteredTypes()Get all registered agent type names
clear()Remove all registrations (for testing)

registry.replace(config) ​

Overwrites an existing static agent registration with a new config, returning whether a prior registration existed. Throws if a factory is registered under the same name.

typescript
const registry = new AgentRegistry();
registry.register(myAgent);

// Later: swap the registered reference (e.g., to attach hooks)
const agentWithHooks = { ...myAgent, hooks: makeHooks() };
const hadPrior = registry.replace(agentWithHooks);
// hadPrior === true

When to use: The Temporal runtime resolves agents BY NAME from the registry — workflow inputs only carry the agent name, not the agent reference. Per-call hooks attached to the agent passed into executor.execute(agent, ...) aren't honored unless the registered reference itself carries them. replace() is the canonical way to swap the reference (most commonly in tests that exercise per-test hooks). JS / DBOS runtimes read the agent reference inline at execute() time and don't need this API.

Don't replace factories. If a factory is registered under the same name, replace() throws. Use unregister() followed by registerFactory() to replace a factory.

Sub-Agent Handling ​

Sub-agents execute as child workflows:

typescript
// In workflow
import { executeChild } from '@temporalio/workflow';

// When parent needs to execute sub-agent
const subAgentResult = await executeChild('agentWorkflow', {
  args: [
    {
      agentType: 'analyzer',
      sessionId: `${parentSessionId}-sub-${callId}`,
      streamId: parentStreamId, // Same stream for unified streaming
      message: inputMessage,
      parentSessionId,
    },
  ],
  workflowId: `agent__analyzer__${subSessionId}`,
  taskQueue: 'agent-tasks',
});

Benefits of child workflows:

  • Independent retry policies
  • Separate timeouts
  • Can be cancelled independently
  • Full workflow history preserved

Remote Sub-Agent Handling ​

Remote sub-agents are executed via a dedicated executeRemoteSubAgentCall activity, separate from both regular tool calls and local sub-agent child workflows.

The activity provides:

  • Deterministic session IDs — {parentSessionId}-remote-{toolCallId} for idempotent restarts
  • Crash recovery — On activity retry, checks transport.getStatus() to avoid re-executing completed agents
  • Heartbeat-based reconnection — Heartbeats store lastSequence so retries can resume the SSE stream mid-flight
  • Stream proxying — Remote agent chunks flow through the parent's stream
  • SubSessionRef tracking — Registers with remote: { streamId, lastSequence } metadata
  • Interrupt propagation — Racing pattern propagates interrupts to abort the remote call
  • Trace propagation — An entry's telemetryParent (EntryTraceOptions on execute, resume and retry) is recorded in the workflow input as traceParent (serialized TraceIds). The activity pins its transport to it (core transportFor), covering the dispatch and the completion fold's reads, so the remote child joins the caller's trace. Every child workflow (local sub-agents and companions) inherits it. No tracer provider is needed, and Temporal emits no spans of its own yet (FU-OTEL-TEMPORAL). The field rides input payloads, which Temporal's determinism check does not compare, so it needs no patched() guard; a workflow without it behaves as before.

See Remote Agents Guide for setup and configuration.

Persistent Sub-Agent Handling ​

Persistent sub-agents in the Temporal runtime execute as child workflows managed through companion tools:

  • Blocking spawn: Uses executeChild() to run the child as a Temporal child workflow. The parent workflow waits for the child to complete. After completion, the workflow calls markPersistentChildStatus to update the SubSessionRef.
  • Non-blocking spawn: Starts a child workflow with parentClosePolicy: 'ABANDON'. The parent continues immediately.
  • Companion tool execution: All companion tool calls route through a dedicated executeCompanionToolCall activity, separate from regular tool execution.

Unlike the JS runtime, the Temporal runtime does not store companion tool results as separate ToolResultMessage entries in the state store. Results flow through the workflow execution context.

Persistent children are tracked via SubSessionRef entries with mode: 'persistent' and are cleaned up when the parent workflow completes.

Re-spawning a completed persistent child continues it on its preserved session (memory retained) rather than recreating it — see Re-consulting a persistent companion (the critic loop).

  • Continuation marker. A consult that reopens a completed child (completed → active, one version-pinned write, so one consult wins) records a pending-continuation marker in the child session's metadata under __helix.companionContinue. The child's continuation entry clears it once its run commits; a continuation that fails before its run commits rolls the reopen back, so the child is never stranded active. The key is visible to listSessions metadata filters. The continuation runs in its own child workflow, …__persistent__<childSessionId>__continue__<toolCallId>.
  • Detached children. A non-blocking child keeps its content in its own session. Its live chunks that reach the parent's stream after the parent's run ended or paused that stream are dropped instead of failing the child; its content stays in its own session and reaches the parent through the snapshot and the next turn's notification.

Known companion gaps on Temporal (CP-41, CP-72, CP-73, CP-105)

  • companion__sendMessage to a running child appends the message to the child's live run and records no stop (WA MR2); the child's workflow loads the history at its next step. The CP-41 record stays open until its cells are re-run on Temporal.
  • Two sendMessage calls to one interrupted child in the same step compute the same child workflow id. If the two starts overlap, the second fails with only a log line (CP-72).
  • When a child ends interrupted, its parent-side ref stays running, so listChildren / getChildStatus report running (CP-73). An untimed companion__waitForResult no longer hangs on it: without a liveness probe it also checks the child's session and that it has no live run, and returns interrupted after three polls. Known bug: a same-step sendMessage resume whose child takes more than about 2 s to reach its run-start commit can be reported interrupted (FU-TEMPORAL-WAITFORRESULT-RESUME-RACE).
  • A sendMessage resume of an interrupted child writes the parent's ref running only in the child's own run-start commit (companionResume, StartRun.resumedCompanionRef; an entry with companionResume and no parentSessionId fails validation_error before any write), and a resume the worker's adapter cannot finish is refused in the parent's companion activity as a tool error (CP-103, fixed). But a resume whose child loses its run-start to another entry, or whose parent ref is gone, still returns delivered: true (CP-105). In each of these cases the resume writes no ref, so the ref keeps its prior status: the stale running of CP-73, since Temporal never records interrupted on a ref.

See the conformance findings.

Activity Configuration ​

The packaged agentWorkflow proxies its activities itself, so their timeouts and retry policy are fixed; neither the executor nor the worker configures them:

  • Every activity except the terminal finalization: startToCloseTimeout: '5 minutes', retry initialInterval: '1 second', backoffCoefficient: 2, maximumInterval: '10 seconds', maximumAttempts: 3.
  • The terminal finalization (finalizeTerminalRun: the run's terminal status and the stream end, after the terminal commit) runs on its own budget: unlimited attempts within 3 minutes (scheduleToCloseTimeout), each attempt at most 1 minute.
  • A remote sub-agent call (executeRemoteSubAgentCall) also gets a 30-second heartbeat timeout and a start-to-close timeout sized from the call, with the same 3-attempt retry policy.

The 3-attempt budget is what bounds a forced call's retryable provider errors (see Completion strategies).

Error Classification ​

When LLM calls fail during Temporal activities, errors are classified using the same HelixError system as other runtimes. The classification has to cross the Temporal serialization boundary:

  1. Activity -- runLLMStep classifies the failure. The adapter's onError is held (core holdLLMErrorReport) until the step's outcome is known: an error chunk (with code and recoverable) is written only for a terminal failure, never for an attempt that an activity retry or a recoverable error result will re-run, because a client stops reading at an error event. The held report is written as the activity's last step, after the attempt's usage recording, afterLLMCall / onMessage hooks and other writes, so an attempt retried because one of them threw writes no error chunk.
  2. Workflow -- A terminal failure ends the run through the terminal order: the terminal commit with failed and the error, the run status, then the failed stream. The activity computes the step's classified ErrorDetail while the real error exists and returns it (terminal.errorDetail), so an ordinary terminal LLM error keeps its code, category and retryable on AgentResult.errorDetail, the persisted SessionState.errorDetail and the error onAgentFail receives (getErrorDetail(error)). Other activity failures still fall back to the wrapping ActivityFailure's detail (DI-04).
  3. Retries -- A retried activity streams its step again under a new attemptId; the superseded attempt's content is dropped by readers.

See Error Handling Guide for the complete error pipeline.

A blocked, refused or truncated final reply completes the run (DI-26)

For an agent without outputSchema, the workflow never classifies a text step's stop reason. A final step that stops with content_filter, refusal, max_tokens, error or unknown ends the run completed with no error, where JS and both Cloudflare runtimes fail it. Observed live for content_filter and max_tokens; the other reasons follow from source. Agents with an outputSchema are not affected. See the error handling guide and DI-26 in the conformance findings.

Completion strategies ​

Temporal runs completion strategies like the other runtimes, so an outputSchema agent finishes on Claude Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1 (through native_output). The worker's LLM adapter decides; the client executor has none.

  • Per call, in the runLLMStep activity. The activity resolves the model's capabilities on every LLM call and, on a forced call, builds the directive for the strategy it selects (forced_tool or native_output). Temporal never runs llmConfigOverride, so the agent's configured model is the forced call's model. A native_output call streams no text_delta; hook payloads (completionStrategy, capabilityProfile) and the attempt_started chunk (strategy, capabilityProfile) record what was applied. The activity also decides the forced call's outcome and returns it to the workflow, which applies it.
  • Early refusal. An outputSchema agent whose model has no strategy, or whose completion schema the selected native_output model cannot take, is refused in the initializeAgentState / startRunCommit activity, before its run-start commit: execute(), resume() and retry() throw a non-retryable HelixError (framework_completion_strategy_unavailable / framework_completion_schema_unsupported). A brand-new session (one with no run yet) is marked failed with the typed detail and its stream is failed (root sessions); an existing session is left as it was. A sub-agent or companion is refused at its own start, so the parent sees the child fail (a refused companion re-consult reports delivered: false with the refusal's code and errorDetail). The check resolves the agent through your agentResolver. Only a registry miss (core AgentResolutionError with code: 'framework_agent_not_found', which AgentRegistry.resolve / get throw; matched with isAgentNotFoundError) skips it: the entry commits its run and fails at its first resolve. Any other resolver error, such as an agent factory that throws, fails the initializeAgentState / startRunCommit activity (or, for a companion sendMessage resume, the companion activity) before anything is committed or any child starts. A custom resolveAgent should throw AgentResolutionError.notFound(type, availableTypes) for an unknown type.
  • Provider errors on a forced call. A retryable or unclassified error is rethrown so the activity retry policy (maximumAttempts: 3) repeats the same forced attempt without consuming one; when the retries run out, the run fails with the provider's detail and its forced phase is closed failed in the terminal commit. A non-retryable error fails the run and the phase at once. afterLLMCall fires once per attempt, so it can fire more than once for one forced attempt. The hook's result does not carry onErrorFailure here: the held error report is written after the hook, so a failed error-chunk write is only known afterwards (logged, and set on the returned step result).
  • Prompt-prefix check. On a model whose thinking is bound to history, the activity fingerprints the request and refuses a changed system prompt or tool set with framework_prompt_prefix_changed (Preserved thinking) before the attempt's step_start, so a refused attempt writes no attempt chunk. The assistant message records the fingerprint (metadata.helixPrefix).
  • Hook order. beforeLLMCall fires before the attempt's step_start (it used to fire after it), so a chunk a hook emits with ctx.emit now precedes that step_start.
  • Terminal LLM errors keep their classified errorDetail (code, category, retryable) on the result, the persisted session and onAgentFail (DI-24, see Error Classification).

Deploying

Every workflow-side decision is gated by wf.patched('helix-completion-strategy-v1'). A workflow started before the upgrade that reaches a gated point while running on an upgraded worker records the patch marker and takes the new path from there on; a replay of its earlier history keeps the old path, so recorded workflows replay deterministically. The exposure is the rollout overlap, while old and new workers both poll the task queue. A workflow task processed by an old worker that receives a runLLMStep result produced by a new worker classifies it the old way: a forced call's non-retryable provider error is then written as an error chunk by the new activity but counted as a miss by the old workflow code, so such a run may show an extra error chunk and can end framework_forced_completion_exhausted. An old worker that picks up a workflow task of a workflow that already took the new path fails that task with a nondeterminism error, which Temporal retries until a new worker takes it. A new workflow's runLLMStep executed by an old worker gets the legacy request, which on a 5.5-generation model can end framework_forced_completion_exhausted as before the upgrade. Once every worker runs this version none of this applies. (An upgraded worker that receives a result from an older activity also classifies it the legacy way.) Keep the patch until every workflow started before the upgrade has finished (FU-TEMPORAL-COMPLETION-PATCH-DEPRECATE). The one workflow-side change that is not gated is an activity argument: a retry() workflow passes retryEntry: true to initializeAgentState, so the retried run keeps the forced phase startRunCommit restored. Replay is unaffected, because Temporal matches commands on activity type and id, never on arguments. During the overlap it can mix versions: a retry whose startRunCommit ran on an old worker restores the target's raw phase (without the new correction for a forced step's own transition commit), and a new worker's initializeAgentState then keeps it, so a retry after an exhausted forced phase re-forces the spent attempt once (and a budget-0 terminal_text / max_tokens retry restores no phase instead of failed); when the hook half runs on an old worker, or the workflow task runs old code (which passes no argument), the phase is reset as before the upgrade and the retried run starts with a work call.

Crash Recovery ​

Temporal provides automatic crash recovery:

mermaid
graph TB
    W1["Worker 1 starts workflow"]
    W1 --> S1["Step 1 completes, state saved"]
    S1 --> S2["Step 2 completes, state saved"]
    S2 --> Crash["Worker 1 crashes"]

    Crash --> Detect["Temporal detects failure"]
    Detect --> Resched["Workflow task rescheduled"]

    Resched --> W2["Worker 2 picks up"]
    W2 --> Replay["Replays history (deterministic)"]
    Replay --> Continue["Continues from Step 3"]
    Continue --> Complete["Completes normally"]

Key points:

  • Workflow code must be deterministic
  • State is reconstructed from event history
  • Activities are not re-executed (results cached)

Interrupt and Resume ​

The Temporal runtime supports soft interruption with the ability to resume later:

typescript
// Interrupt the agent (soft stop): observed before its next step's LLM call
await handle.interrupt('user_requested');

// The run ends 'interrupted' once the workflow has observed it
const interrupted = await handle.result();
console.log(interrupted.status); // 'interrupted'

// Later, resume execution
const { canResume } = await handle.canResume();
if (canResume) {
  const newHandle = await handle.resume();
  const result = await newHandle.result();
}

Handles do not pass resume options on (CP-31)

An interrupted run ends the session's stream. The resume's run-start winner now claims the stream and makes it live again before it writes, as every entry does (it used to leave the stream ended, so the resumed turn could not write, CP-76; that record stays open until its conformance cell lands). executor.resume() honours mode and message (see Resume modes above), but no handle passes them on: the execute() handle's and the factory handle's resume() throw, and a getHandle() handle calls executor.resume(agent, sessionId) without its options (CP-31). Call executor.resume(agent, sessionId, options) directly. See the conformance findings.

Workflow Setup (v7) ​

The v7 stateless workflow exits at every HITL boundary and the executor spawns a fresh __resume-<runId> workflow on each resume. Its INTERRUPT_SIGNAL_NAME handler only wakes the running execution; the run's durable stop record in the state store is the canonical channel (see Interrupts and sub-agents). Register agentWorkflow directly, or re-export it under a stable name your TemporalAgentExecutor.workflowName expects:

typescript
import { agentWorkflow as v7AgentWorkflow } from '@helix-agents/runtime-temporal/workflow';
import type { AgentWorkflowInput, AgentWorkflowResult } from '@helix-agents/runtime-temporal';

// Re-export under your workflow name. The imported `agentWorkflow` sets up
// `proxyActivities<GenericActivities>`, the INTERRUPT_SIGNAL_NAME handler,
// and child-workflow dispatch internally. The wrapper exists only to
// register the function under a stable name.
export async function agentWorkflow(input: AgentWorkflowInput): Promise<AgentWorkflowResult> {
  return v7AgentWorkflow(input);
}

Removed v6 signals

v7's stateless suspension model removed four Temporal signals: submitToolResult, childSuspended, childWoke, runResumed. Code that called handle.signal('submitToolResult', ...) or used the RUN_RESUMED_SIGNAL_NAME constant must migrate to the v7 stateless contract — submission writes to durable state, and executor.resume() spawns a fresh __resume-<runId> workflow that drains submissions via the applyResultsAndReload activity. The two-arg runAgentWorkflow(input, activities, options) signature is also gone; the v7 export is agentWorkflow(input). See the v6 → v7 upgrade guide. INTERRUPT_SIGNAL_NAME ('agent_interrupt', exported from @helix-agents/runtime-temporal/workflow) is the one signal the workflow still handles: the executor's handles send it, carrying the recorded StopRequest.

Interrupts and sub-agents ​

An interrupt stops the run it is sent to, at a step boundary:

  1. Request - Every handle's interrupt() (the execute(), resume(), retry() and getHandle() handles) stops the handle's own run through core stopRun (WA MR2): the stop is recorded on the run, then, while the run's workflow execution is open, signalled (INTERRUPT_SIGNAL_NAME, carrying the StopRequest). A run whose workflow is closed (a suspended run) is ended by the call itself. A run that already ended resolves the call (logged). The signal only wakes the workflow; the run's durable stop record reaches it from any process, and the workflow reads it with the readStopActivity activity.
  2. Observation - The workflow reads the record at every main-loop iteration, before that step's LLM call, and on a resume, right after the drain of submitted results (applyResultsAndReload), before it decides whether the run stays suspended. It is never checked while a step's LLM call, tools or sub-agent children are in flight, so they finish first. Every live-run commit is guarded (refuseIfStopped), so a stop that lands before a commit ends the run by the stop instead.
  3. Stop - The run settles pending client-tool calls with an aborted tool_end, commits its terminal (interrupted; an abort: failed with framework_cancelled), then writes run_interrupted and returns that committed outcome. The terminal is announced only after it commits (WA MR2 F4-C S3).

abort() also cancels the workflow, and resolves once the stop is durable (recorded, the workflow signalled and cancelled; F4-C m4). It never waits for the workflow to close. The cancelled workflow commits its own terminal (its terminal runs non-cancellable), and result() reports it. After stopRun answers, abort() makes one bounded owner check: a run still live under a workflow that is ALREADY closed at that check is settled by the call itself. A workflow that finishes closing just after the check (for example, one completing its suspension when the cancel lands) is not: a handle reading its result() settles it, else it waits for FU-TEMPORAL-STOP-SETTLE-CLOSED-OWNER. A failure of that check is logged and never rejects abort() (the stop is already durable). agent-server's stop routes, which call the handle, answer at once too.

A terminal that did not commit (F4-C S3). When a run's K5 fails for any reason other than a lost claim, the workflow retries it once in its top-level catch. If that also fails, the workflow fails with the typed temporal_terminal_not_committed failure; it never returns a synthesized interrupted / failed result over a session that is still active. The handle's result() then reconciles the run from the store, because the owner is now closed:

  • a run that already ended, or whose own K5 landed, reports that outcome;
  • a recorded stop gets the stop's terminal;
  • otherwise the run fails with the failure it was ending with (core failCommittedRun).

If that settle does not land either, result() rejects with its error and caches nothing, so a later result() retries.

A handle whose workflow ended suspended although its run's stop was recorded settles the run by that stop and reports the stopped outcome. Two cases are still open:

  • a stopped run whose workflow closed with no handle reading its result stays live until the next entry settles it (FU-TEMPORAL-STOP-SETTLE-CLOSED-OWNER);
  • a stop recorded after a suspension commit, with no handle reading, is settled only by the next continuation (FU-TEMPORAL-STOP-AFTER-SUSPENSION-COMMIT).

Child workflows are not signaled. On an interrupt, a sub-agent child the step is waiting on runs to its own end before the parent sees the stop. On an abort, the parent workflow's cancellation reaches the children it is waiting on (Temporal's child cancellation): each child is cancelled with it and commits its own terminal, failed with the cancellation's message and no framework_cancelled code, because the child has no stop record of its own (FU-TEMPORAL-CHILD-CANCEL-UNTYPED-FAILURE). A non-blocking persistent companion (started with ParentClosePolicy.ABANDON) keeps running either way. Temporal does not record 'interrupted' on a companion's SubSessionRef (Sub-Agents).

See Interrupt and Resume for complete documentation including resume modes and error handling.

Stream Resumption ​

The Temporal runtime supports stream resumption for client disconnections and page refreshes with the same @helix-agents/ai-sdk handlers as the other runtimes: buildSnapshot for the snapshot route and handleChatStream for the chat route.

Setting Up Stream Resumption ​

typescript
import { buildSnapshot, handleChatStream } from '@helix-agents/ai-sdk';

const deps = { executor, stateStore, streamManager, agent: myAgent }; // TemporalAgentExecutor

// GET /api/chat/:sessionId/snapshot
const snapshot = await buildSnapshot(deps, { sessionId });

// POST /api/chat/:sessionId (send or resume)
return handleChatStream(deps, { sessionId, messages, request });

Frontend Integration ​

Seed the chat from the snapshot and resume only while the run is live, with snapshotResumeOptions:

typescript
import { snapshotResumeOptions } from '@helix-agents/ai-sdk/client';

const opts = snapshotResumeOptions(snapshot);
// opts.messages: the seed (without the in-flight message when resuming)
// opts.resume: true only for status 'active' / 'paused'
// opts.resumeFromSequence / opts.existingMessageId / opts.replayPrefix: the reconnect target

A snapshot is taken from one checkpoint: its messages are exactly the rows committed up to its streamSequence cursor, and a resume streams everything after that cursor (the server replays the in-flight message's committed prefix first). There is no partial-content merge and no content-replay option any more. See Resumable Streams and the upgrade guide for the full client wiring.

Key Considerations ​

  • Durable stream manager - Stream resumption relies on a stream manager that stores chunks with sequence numbers (for example RedisStreamManager)
  • Chunk retention - Configure appropriate TTL for stream chunks based on expected session duration
  • Multi-worker - Works across multiple workers since stream state is in the shared store
  • Retried activities - A retried activity re-streams its step under a new attemptId; resumed readers drop the superseded attempt

Concurrency Protection ​

The run-start commit is the single gate (see Run Start and Ownership); workflow ids add a second guard:

  • Every execute() turn of a session runs under the session's base workflow id, and Temporal refuses a duplicate while one is running
  • Concurrent execute() calls result in workflow collision errors (AgentAlreadyRunningError)
  • resume() / retry() run under per-run ids (__resume-<runId>, __retry__<runId>); for them the run-start commit alone picks the winner, and a loser writes nothing

Those collision errors are not only for concurrent calls: an execute() for the next turn made right after the previous turn reads completed, but before its workflow has closed, collides too (CP-84, see Multi-Turn Conversations).

resume() against a live run is refused (CP-83)

A session that reads active with no pending calls and whose current run's workflow is still running is a live run: resume() refuses it in every mode with AgentAlreadyRunningError before any write, the rule DBOS enforces. A session whose run's workflow closed (a crashed or terminated execution) is not refused: the resume's run-start commit supersedes the stale run. In earlier releases each such resume started a new __resume-N workflow beside the live one, and the live run ended failed with its answer lost. See CP-83 in the conformance findings.

retry() Method ​

Retry failed agents in Temporal:

typescript
const result = await handle.result();
if (result.status === 'failed') {
  const retryHandle = await executor.retry(agent, sessionId);
  // or
  const retryHandle = await handle.retry();
}

The retry creates a new workflow execution (…__retry__<runId>) whose entry is one run-start commit: it rewinds to the retry target and appends the input atomically. The target is core planRetry, the same on every runtime: by default the failed run's last committed step (pass message, or the retry throws validation_error), or, when the run failed on its first step, its run-start commit (the triggering message is re-run, followed by message if given). The terminal checkpoint a failure writes is never the target. retry({ checkpointId }) resolves the checkpoint before any write: an unknown id throws state_checkpoint_not_found and another session's id throws state_checkpoint_foreign, with the session left untouched. See JS runtime: retry() for the full rules.

The retried run restores the target's customState, stepCount and forced-completion phase, as on runtime-js. A retry into a pending forced phase continues the restored attempt: the retried run's first call is the forced call, resolved for the adapter the worker has now. A default retry whose target is the forced step that recorded the model's miss on the phase's last attempt (under either strategy: a text, invalid or protocol-violating answer to a __finish__ forced call, or a native_output call's prose or schema-invalid answer) restores the phase as that step's own forced transition left it, so that attempt is not re-forced. Only a miss the forced step recorded is spared: every other failure inside a forced phase restores it pending and re-forces the attempt, as on runtime-js. For example, a finishWith failure (the default target is the phase-2 append commit, which holds the phase pending), or a provider error, retryable or not (the forced step records no row, so the target is the commit before it, also pending) (FU-COMPLETION-RETRY-AFTER-FINISHWITH-EXHAUSTION). A maxCompletionRetries: 0 run that failed at a terminal_text or max_tokens trigger restores a failed phase, one that failed at a max_steps trigger restores none, as on runtime-js. (Before @helix-agents/runtime-temporal 4.0 the retry's entry reset the phase, so a pending phase was dropped and the retried run's first call was a work call.) A retry() against an outputSchema agent whose model has no completion strategy is refused before its run-start commit (see Completion strategies).

Branching (execute(agent, input, { branch })) now honours branch.checkpointId and branch.messageIndex (CP-33): the new session gets exactly the checkpoint's first messageCount messages and its customState, or the first messageIndex messages. A foreign or unknown branch.checkpointId is rejected by execute() itself, before startWorkflow.

Retrying to a finishWith step's checkpoint deletes the call's result (RM-52)

The workflow commits a step (its step checkpoint) before runPhase2FinishWith runs, and the finishWith result is appended afterwards by its own append commit. So the step checkpoint of a step that called a finishWith tool ends on the tool_use without its result. It is no longer the turn's latest checkpoint (the append and terminal commits follow it), and a default retry() never targets it, but a retry({ checkpointId }) that names it truncates the transcript there: the call's real result is deleted and its tool_use is left unpaired (observed on the memory store before this release). Do not retry to the checkpoint of a finishWith step. See RM-52 in the conformance findings.

The handle executor.resume() returns now settles result() on the resumed workflow's own result, as the execute() handle does, and the terminal commit (with failed) is written before the workflow returns. It used to settle on the first error chunk, before the failure was stored, so an immediate retry() could throw (CP-80; the record stays open until its conformance cell lands; see the conformance findings).

Method Comparison ​

MethodPurposeStream BehaviorValid From Status
execute()New/continue conversationResets streamAny except running
resume()Continue after interruptPreserves streaminterrupted, paused (incl. HITL suspension); any status with from_checkpoint once its run is not live
retry()Recover from failureResets to its targetfailed

Determinism Requirements ​

The packaged agentWorkflow is deterministic: all of its I/O runs in activities. Workflow code of your own must be deterministic too:

typescript
// BAD - Non-deterministic
export async function myWorkflow(input) {
  const timestamp = Date.now(); // Different on replay!
  const random = Math.random(); // Different on replay!
  const uuid = crypto.randomUUID(); // Different on replay!
}

// GOOD - Use Temporal APIs
import { sleep, uuid4, workflowInfo } from '@temporalio/workflow';

export async function myWorkflow(input) {
  const info = workflowInfo();
  const timestamp = info.startTime; // Deterministic
  const id = uuid4(); // Deterministic (seeded)
  await sleep('5 seconds'); // Deterministic timer
}

Move non-deterministic operations to activities:

  • LLM API calls
  • Database queries
  • External API calls
  • Random number generation

Observability ​

Temporal Web UI ​

Access at http://localhost:8233 (local) or via Temporal Cloud.

View:

  • Workflow history and events
  • Activity execution details
  • Pending/failed workflows
  • Search by workflow ID or type

Workflow Queries ​

agentWorkflow registers no query handlers: a run's progress lives in the session store, not in workflow state. Read it with handle.getState() (status, step count, custom state) or follow the stream.

Production Deployment ​

Worker Scaling ​

Run multiple workers for throughput:

bash
# Scale horizontally
docker-compose scale worker=5

Workers pull from the same task queue - Temporal handles distribution.

Temporal Cloud ​

For production, use Temporal Cloud:

typescript
import { Connection, Client } from '@temporalio/client';

const connection = await Connection.connect({
  address: 'your-namespace.tmprl.cloud:7233',
  tls: {
    clientCertPair: {
      crt: fs.readFileSync('client.pem'),
      key: fs.readFileSync('client.key'),
    },
  },
});

Monitoring ​

Set up metrics:

typescript
import { Runtime } from '@temporalio/worker';

Runtime.install({
  telemetryOptions: {
    metrics: {
      prometheus: { bindAddress: '0.0.0.0:9464' },
    },
  },
});

Limitations ​

Workspaces not supported ​

The Temporal runtime does NOT wire workspace providers. execute() / resume() / retry() of an agent that declares a workspace fail fast at run start with an error of the form:

Temporal runtime does not support workspaces (yet — design-reserved).
Agent '<agentType>' declares a workspace.
Use one of: JS runtime, Cloudflare Durable Object runtime via @helix-agents/agent-server.

The check (assertRuntimeSupportsWorkspaces, shared with the Cloudflare Workflows runtime) runs synchronously at the top of each entry point, before any state load or workflow start, so no side effects occur when it rejects. describeCapabilities() returns an empty workspaceProviderKinds array so AgentServer's executor-side check catches a workspace-declaring agent at construction time too.

If your agent needs files, shell, code, or snapshots, run it on the JS runtime or the Cloudflare DO runtime via @helix-agents/agent-server (both DO wire workspaces — see the workspaces overview for provider options). Workspace support on Temporal is design-reserved for a future plan.

stopWhen is ignored and a maxSteps cut-off completes ​

stopWhen is never evaluated on Temporal (DI-29)

The workflow calls the stop check (shouldStopExecution) with maxSteps only, so an agent's stopWhen predicate never runs: after it would have fired, the agent keeps calling the LLM and running tools until the model stops on its own or maxSteps is reached (50 when the agent sets none; a sub-agent or companion child workflow is not capped, FU-TEMPORAL-CHILD-MAX-STEPS). Observed live (a stopWhen: () => true agent made 3 LLM calls where JS made 1). JS, the Cloudflare DO and Cloudflare Workflows honour it. Use a finishWith tool or an outputSchema to end the loop. See the stopWhen guide and DI-29 in the conformance findings.

A maxSteps cut-off completes on Temporal (DI-25)

An agent without outputSchema that runs out of maxSteps mid-tool-loop ends completed with no output and no error code here. JS, the Cloudflare DO and Cloudflare Workflows fail it with framework_max_steps_exhausted and let retry() continue it; on Temporal retry() refuses the session because it is not failed. Wiring core resolveStopDecision and the run's completionReason into the workflow is FU-CFD1C-DI29-DI25-TEMPORAL (CF-D1 MR 2). See When maxSteps runs out.

Memory auto-injection/extraction not supported ​

The Temporal runtime does not wire any MemoryManager logic. The memory: config field on an agent definition is accepted at the type level but is a silent no-op — no memory injection into prompts and no extraction after tool calls. Agents that need live memory recall or storage must run on the JS or Cloudflare runtimes.

LLMConfig.cache strategies are applied on the Temporal runtime (as on the JS, Cloudflare, and DBOS runtimes).

Higher Latency ​

Each activity invocation adds network overhead. Batch operations when possible.

Determinism Constraints ​

Workflow code restrictions can be challenging. Move all I/O to activities.

Infrastructure Overhead ​

Requires running Temporal server and workers alongside your application.

Learning Curve ​

Temporal concepts (workflows, activities, replay, determinism) require understanding.

Best Practices ​

1. Keep Workflows Thin ​

agentWorkflow only orchestrates; every read and write is an activity. Do the same in workflows of your own:

typescript
// Workflow just coordinates
export async function myWorkflow(input) {
  const state = await loadState(input.sessionId);
  const result = await executeStep(state); // Activity does the work
  await saveState(state);
}

2. Appropriate Timeouts ​

agentWorkflow sets its own activity timeouts (see Activity Configuration). In a workflow of your own (for example one that starts agentWorkflow as a child and does other work around it), give your activities realistic timeouts:

typescript
const { fetchCustomerRecord } = proxyActivities<typeof myActivities>({
  startToCloseTimeout: '5 minutes',
});

3. Keep Tools Within the Activity Budget ​

Each server tool runs in an executeToolActivity activity under agentWorkflow's fixed policy: a 5-minute start-to-close timeout and 3 attempts, with no heartbeat timeout (see Activity Configuration). A tool that can run longer should start the work elsewhere (for example a workflow of your own) and return a handle to it. In activities of your own that set a heartbeatTimeout, heartbeat while they work:

typescript
import { heartbeat } from '@temporalio/activity';

export async function processItems(items: Item[]): Promise<void> {
  for (const item of items) {
    await processItem(item);
    heartbeat(); // Report progress
  }
}

4. Histories Stay Per Run ​

agentWorkflow does not continue-as-new. Every entry (execute() turn, resume(), retry()) starts a new workflow execution, so one execution's history holds one run's steps. A long-lived workflow of your own should reset its history with continueAsNew:

typescript
import { continueAsNew, workflowInfo } from '@temporalio/workflow';

export async function myLongWorkflow(input) {
  if (workflowInfo().historyLength > 10000) {
    await continueAsNew<typeof myLongWorkflow>(input);
  }
  // ...
}

Next Steps ​

Released under the MIT License.