Temporal Runtime
The Temporal runtime (@helix-agents/runtime-temporal) executes agents as durable Temporal workflows. This provides crash recovery, automatic retries, and production-grade reliability for long-running agent tasks.
When to Use
Good fit:
- Production workloads requiring reliability
- Long-running agents (hours or days)
- Agents that must survive process restarts
- Complex multi-agent orchestrations
- Operations requiring audit trails and observability
Not ideal for:
- Quick development iteration (infrastructure overhead)
- Simple, short-lived agents
- Cost-sensitive deployments without existing Temporal infrastructure
Prerequisites
You need a running Temporal server:
Option 1: Temporal Cloud (Recommended for production)
# Sign up at https://temporal.io/cloudOption 2: Local development
# Using Docker
docker run -d --name temporal \
-p 7233:7233 -p 8233:8233 \
temporalio/auto-setup:latest
# Or using Temporal CLI
temporal server start-devInstallation
npm install @helix-agents/runtime-temporal @helix-agents/store-redis @temporalio/client @temporalio/worker @temporalio/workflow @temporalio/activity zodThe @temporalio/* peer dependencies are ^1.13.0 (they were ^1.11.0), and zod (^4.1.8) is a peer dependency too.
Architecture
graph TB
subgraph App ["Your Application"]
Executor["<b>TemporalAgentExecutor</b><br/>Starts workflows · Returns handles<br/>Reads from StateStore/StreamManager"]
end
subgraph Server ["Temporal Server"]
ServerDesc["Persists workflow state<br/>Manages task queues<br/>Handles retries and timeouts"]
end
subgraph Worker ["Temporal Worker"]
subgraph Workflow ["Agent Workflow"]
WF["Orchestrates execution<br/>Calls activities for LLM/tools"]
end
subgraph Activities ["Activities"]
Act["LLM calls · Tool execution<br/>State persistence"]
end
end
App --> Server --> WorkerSetup Guide
1. Create the Workflow
Register the framework's agentWorkflow under the name your executor's workflowName expects:
// src/workflows.ts
import { agentWorkflow as runAgentWorkflow } from '@helix-agents/runtime-temporal/workflow';
import type { AgentWorkflowInput, AgentWorkflowResult } from '@helix-agents/runtime-temporal';
export async function agentWorkflow(input: AgentWorkflowInput): Promise<AgentWorkflowResult> {
return runAgentWorkflow(input);
}The workflow proxies every GenericActivities method, runs the step loop, the run-start commit, the suspension and terminal commits, and child-workflow dispatch.
2. Create Activities
Activities perform the actual work (LLM calls, tool execution, state and stream writes). Configure one GenericActivities instance and register every method on its prototype:
// src/activities.ts
import { AgentRegistry, GenericActivities } from '@helix-agents/runtime-temporal';
import { VercelAIAdapter } from '@helix-agents/llm-vercel';
import { RedisStateStore, RedisStreamManager } from '@helix-agents/store-redis';
const registry = new AgentRegistry();
registry.register(ResearchAgent);
registry.register(AnalyzerAgent);
export function createActivities(): Record<string, (...args: never[]) => unknown> {
const activities = new GenericActivities({
stateStore: new RedisStateStore(redisClient),
streamManager: new RedisStreamManager(redisClient),
llmAdapter: new VercelAIAdapter(),
agentResolver: {
resolveAgent: (type) => registry.resolve(type, { env: undefined, sessionId: '', runId: '' }),
},
});
// Bind every prototype method: the workflow calls activities a hand-written
// list would miss (`Activity function <name> is not registered on this Worker`).
const proto: object = Object.getPrototypeOf(activities);
return Object.fromEntries(
Object.getOwnPropertyNames(proto)
.filter((name) => name !== 'constructor')
.map((name) => [name, Object.getOwnPropertyDescriptor(proto, name)?.value] as const)
.filter(
(e): e is readonly [string, (...args: never[]) => unknown] => typeof e[1] === 'function'
)
.map(([name, fn]) => [name, fn.bind(activities)] as const)
);
}This release adds activities (the run-start, suspension and terminal commits, among others), so a worker that registers a hand-written list of activity names must be updated.
Optional GenericActivities dependencies for ownership and attempts:
| Option | Default | Purpose |
|---|---|---|
ownerProbe | describe() via the activity context | Answers whether a live run's owner execution (${workflowId}:${workflowRunId}) is still running. Tests inject one |
ownerWait | 30 s, polling from 100 ms | Bounds an entry's wait for a suspending owner workflow to close ({ timeoutMs, initialIntervalMs }) |
workflowProbe | describe() via the activity context | Describes a workflow by id; used to recognise a stalled companion continuation |
llmAttemptFenceIntervalMs | 2 000 | How often a running runLLMStep attempt heartbeats and checks for a newer attempt of its step (a superseded attempt aborts) |
The default probes use the activity context's client, which needs @temporalio/activity 1.13 or later.
3. Create the Worker
The worker processes workflows and activities:
// src/worker.ts
import { Worker, NativeConnection } from '@temporalio/worker';
import { createActivities } from './activities';
async function runWorker() {
const connection = await NativeConnection.connect({
address: process.env.TEMPORAL_ADDRESS ?? 'localhost:7233',
});
const worker = await Worker.create({
connection,
namespace: 'default',
taskQueue: 'agent-tasks',
workflowsPath: require.resolve('./workflows'),
activities: createActivities(),
});
await worker.run();
}
runWorker().catch(console.error);4. Create the Executor
The executor starts workflows and returns handles:
// src/executor.ts
import { Client, Connection, WorkflowNotFoundError } from '@temporalio/client';
import { TemporalAgentExecutor } from '@helix-agents/runtime-temporal';
import { RedisStateStore, RedisStreamManager } from '@helix-agents/store-redis';
async function createExecutor() {
const connection = await Connection.connect({
address: process.env.TEMPORAL_ADDRESS ?? 'localhost:7233',
});
const client = new Client({ connection });
// Wrap Temporal client to match interface
const temporalClientAdapter = {
startWorkflow: async (name, options) => {
const handle = await client.workflow.start(name, {
workflowId: options.workflowId,
taskQueue: options.taskQueue,
args: options.args,
});
return wrapHandle(handle);
},
// `runId` pins one execution of a reused id (a stop targets the run's own execution).
getHandle: (workflowId, runId) => {
return wrapHandle(client.workflow.getHandle(workflowId, runId));
},
// REQUIRED: the liveness of ONE workflow execution (see "Run start and ownership").
describeWorkflow: async (workflowId, runId) => {
try {
const description = await client.workflow.getHandle(workflowId, runId).describe();
return { status: description.status.name }; // 'RUNNING', 'COMPLETED', ...
} catch (error) {
if (error instanceof WorkflowNotFoundError) return null; // closed and past retention
throw error;
}
},
};
return new TemporalAgentExecutor({
client: temporalClientAdapter,
stateStore: new RedisStateStore(redis),
streamManager: new RedisStreamManager(redis),
workflowName: 'agentWorkflow',
taskQueue: 'agent-tasks',
// entryOutcomeTimeoutMs: 120_000, // run-start fail-safe (default)
});
}Breaking: TemporalClient.describeWorkflow is required
TemporalClient gained a required describeWorkflow(workflowId, runId) method. It returns the execution's status name, or null when the server no longer knows the execution. The executor uses it to refuse an entry while the current run's workflow is still running, and to finish a run whose workflow closed between its terminal commit and its terminal run status. An adapter without it does not type-check. See the upgrade guide.
Using the Executor
Once set up, usage is identical to other runtimes:
const executor = await createExecutor();
// Execute agent with a string message
const handle = await executor.execute(ResearchAgent, 'Research quantum computing');
// Or with multiple messages (for context injection, file attachments, etc.)
// const handle = await executor.execute(ResearchAgent, {
// message: [
// { role: 'user', content: 'Focus on recent breakthroughs', metadata: { source: 'system' } },
// { role: 'user', content: 'Research quantum computing' },
// ],
// });
// Stream events
const stream = await handle.stream();
for await (const chunk of stream) {
console.log(chunk);
}
// Get result
const result = await handle.result();Conversation History
Fixed: Temporal sent the model no conversation history (RM-10)
Before this release, every Temporal LLM call received only the system prompt. The workflow projected session state through a messages field that SessionState does not have, so the model never saw the user's message, earlier turns or tool results. With a real provider every call failed with messages must not be empty, from the first turn on; with a mock adapter the runs appeared to work. Hook messageIndex values were computed from the same empty array. Upgrade @helix-agents/runtime-temporal and @helix-agents/core together.
runLLMStep now loads the complete, count-checked history inside the activity with core loadLLMHistory and builds the model input from it. History never travels through workflow code or activity payloads (Temporal payload size limits, and every payload is recorded in workflow history). The activity returns only what the step produced:
RunLLMStepResult field | Meaning |
|---|---|
nextState.messages | Always [] |
newMessages | Messages this step appended (assistant message, synthetic results), in order |
baseMessageCount | Length of the persisted history the step was built from (base for absolute hook indices) |
If the store returns fewer rows than its live count (after the loader's bounded re-reads), the activity fails with a non-retryable ApplicationFailure carrying the classified ErrorDetail, and the run ends failed with errorDetail.code === 'state_history_incomplete'. No model call is made and the activity is not retried. If a store call throws while loading (for example a dropped Redis or Postgres connection), the ApplicationFailure is retryable (errorDetail.retryable: true), so the activity retry policy applies: a transient blip is retried and the run proceeds. Only when every attempt fails does the run end failed with the same code.
Deploying this version: drain, and never mix old and new workers
The workflow code and the runLLMStep result shape changed together, with no patched() guard. New workflow code replaying an old result throws TypeError: Cannot read properties of undefined (reading 'length') on every workflow task and the workflow hangs in Running (no history is lost; terminate or reset it). Old workflow code receiving a new result persists nothing from the empty nextState.messages: the assistant message is silently lost and its tool results are orphaned.
- Drain in-flight Temporal workflows before deploying.
- Never run old and new workers on the same task queue: cut over fully, or use Worker Versioning or a new task queue.
- Terminate or reset any workflow stuck on the workflow-task error
Cannot read properties of undefined (reading 'length').
Run Start and Ownership
Every entry (execute(), resume(), retry(), a child or companion turn) starts its run with one run-start commit, written by the workflow's entry activity: appendMessages(sessionId, inputRows, { streamSequence, runId, startRun }). It appends the input rows, applies any rewind (from_checkpoint, retry()), creates the run record and its input checkpoint, all atomically. Nothing is written before it: the only executor-side write is a new session's write-once createSession. Only the winner then claims the stream (a compare-and-set), makes it live and writes run_started; a loser writes nothing and its call throws AgentAlreadyRunningError (RunStartConflictError) or RunStartRejectedError.
- Run ids. The executor mints the run id and passes it in the workflow input, so a replay reads the same value.
- Owner token. A run's
ownerTokenis${workflowId}:${workflowRunId}: one Temporal execution. An entry into a session whose current run is still live is refused while that execution isRUNNING(describeWorkflow); once it is closed (completed, failed, terminated, timed out) the run is ownerless and the next entry supersedes it, so a terminated or crashed workflow never bricks the session. - Fail-safe.
execute()/resume()/retry()wait until their entry has an outcome (its run committed and wroterun_started, or it lost). If none arrives withinentryOutcomeTimeoutMs(default 120 000 ms; for example no worker polls the task queue), the call gives the entry up atomically with the store (abandonRunStart, a tombstone that refuses a late commit), terminates the workflow and throws coreRunStartAbandonedError(transport_error, retryable once the workflow was terminated). If the tombstone cannot be written, it throwsRunStartOutcomeUnknownError(non-retryable: read the session before retrying). Both are exported by@helix-agents/core. - Hooks. A run that fails at its entry (the run-start commit, the stream claim or
run_started) fires no lifecycle hook;onAgentFailfires only for a run whoseonAgentStart/onAgentResumedfired. The start hooks are recorded durably (agent-start-hooks-<runId>): an activity retry replays their writes instead of re-firing them, but a worker crash in the middle of them re-fires them (at-least-once). A start hook that throws fails the entry non-retryably. - Terminal order. A terminal exit (completion, failure, stop) writes its terminal commit (
commitState, astatecheckpoint carrying the status, error and the terminal hooks' writes), then the run's terminal status, then ends or fails the stream. The finalization after the terminal commit runs on a long retry budget,terminalFinalizeTimeoutMsin the workflow input (default 3 minutes). Past it the workflow closes with the committed outcome, and the nextgetHandle()/resume()finishes the run. - Suspension order. The step's frames (
tool_start/tool_approval_request), then the suspension commit with the run'ssuspended_*status, thenpauseStream(run_paused). The stream'spausedstatus means the suspension is committed. - Superseded runs. Every write carries the run id and is fenced to it; a run that was taken over stops with
AgentResult.status === 'superseded'(errorDetail.code === 'state_run_superseded',ApplicationFailure.nonRetryable).onAgentSupersededfires, neveronAgentFail, andonAgentCompleteonly when the supersede landed after it fired and before the terminal commit (thencomplete → superseded, the last hook authoritative). - Attempts. Every chunk of an LLM attempt carries
runIdand anattemptId, andstep_startis the attempt's first chunk. An activity retry re-streams the step under a newattemptId; readers drop the superseded attempt's content. Anerrorchunk is written only for a terminal failure, never for an attempt that will be retried. resumeCountis bumped by the winningresume()/retry()entry, after its run-start commit (not for a child re-spawn).
Deploying this release: drain or terminate in-flight workflows
This release changes the workflow's command order with no patched() guard: a suspension now runs commitSuspendedStep → the suspension marker → pauseSuspendedStream, a resume's first command is startRunCommit, commitTerminalState + finalizeTerminalRun replace persistTerminalState, activities were added, and workflow ids changed (__resume-<runId>, __retry__<runId>, a companion continuation …__continue__<toolCallId>). A workflow started by the previous release cannot replay on the new code (nor the reverse, so a rollback needs the same drain): it may get stuck retrying a nondeterministic workflow task, or be failed. Either way, before you deploy, let in-flight Temporal workflows finish or terminate them, and never run old and new workers on the same task queue. A terminated run's session is then superseded by its next entry. See the deploy checklist.
Multi-Turn Conversations
The Temporal runtime supports the same multi-turn conversation API as the JS runtime using the session-centric model:
Using sessionId
// First message - creates a new session
const handle1 = await executor.execute(agent, 'Hello, my name is Alice', {
sessionId: 'session-123',
});
await handle1.result();
// Continue the conversation - same sessionId
const handle2 = await executor.execute(agent, 'What is my name?', {
sessionId: 'session-123',
});The next turn can be refused just after the previous one reads completed (CP-84)
The terminal path writes the terminal commit (the session reads completed), then the run status, then ends the stream, and the workflow closes only after that. Every execute() turn runs under the session's base workflow id (agent__<name>__<sessionId>), so an execute() for the next turn in that gap is refused with AgentAlreadyRunningError: by the owner check while the run's workflow is still RUNNING, or by the workflow-id collision ("session was claimed by a concurrent request"), although nothing else is running. The same call succeeds once the old workflow closes. Wait for handle1.result() (as above) before sending the next turn; a client that sends as soon as it reads completed should retry the refusal briefly. Observed live. See CP-84 in the conformance findings.
Using handle.send()
const handle1 = await executor.execute(agent, 'Hello', {
sessionId: 'session-123',
});
await handle1.result();
const handle2 = await handle1.send('Tell me more');Using Direct Messages
const handle = await executor.execute(agent, {
message: 'Continue from here',
messages: myExternalMessageHistory,
});Multi-Message Input
The message field in all operations accepts either a string or a UserInputMessage[] array. Multi-message input lets you inject context alongside the user's question, attach files, or send multiple messages in a single execution:
const handle = await executor.execute(
agent,
{
message: [
{
role: 'user',
content: 'Background: user is on the enterprise plan',
metadata: { source: 'system' },
},
{ role: 'user', content: 'What features do I have access to?' },
],
},
{ sessionId: 'session-123' }
);
// Messages can include file attachments
const handle = await executor.execute(
agent,
{
message: [
{
role: 'user',
content: 'Analyze this image',
files: [
{ data: 'data:image/png;base64,...', mediaType: 'image/png', filename: 'chart.png' },
],
},
],
},
{ sessionId: 'session-123' }
);Multi-message input also works with handle.send() and resume with_message mode:
const handle2 = await handle1.send([
{ role: 'user', content: 'Additional system context' },
{ role: 'user', content: 'Follow-up question' },
]);
const resumed = await handle.resume({
mode: 'with_message',
message: [{ role: 'user', content: 'Here is what I meant instead' }],
});String and multi-message inputs can be mixed freely across turns in the same session.
Behavior Table
| Input | Messages Source | State Source |
|---|---|---|
message only (new session) | Empty (fresh) | Empty (fresh) |
message + sessionId (existing) | From session | From session |
message + messages | From messages | Empty (fresh) |
message + state | Empty (fresh) | From state |
message + sessionId + messages | From messages (override) | From session |
message + sessionId + state | From session | From state (override) |
| All four | From messages (override) | From state (override) |
Note: In the behavior table,
messagecan be either astringorUserInputMessage[]. Both forms work identically with all combinations.
Caller-provided
history:prefix: The agent-inputhistory:prefix is honored on fresh sessions — the executor pre-claim no longer drops it, so the seeded history lands in the message log andonMessagefires with the correctmessageIndex. On continuation turns (an existing session with a non-empty durable message log) the prefix is dropped — this is now the uniform behavior across all runtimes (history:is fresh-session seeding only). The drop is observable via atemporal.history_dropped_on_continuationwarn. The earlier runtime-cloudflare divergence (CF appended the prefix on continuations, doubling the durable log) was resolved by gating every runtime on an empty durable log, perFU-CROSS-RUNTIME-HISTORY-CONTINUATION-DOUBLING.
See JS Runtime - Multi-Turn Conversations for detailed documentation.
Execute vs Resume Semantics
The Temporal runtime provides two methods for running agents with different semantics:
| Method | Use Case | Stream Behavior | Session Behavior |
|---|---|---|---|
execute() | Start new conversation or add new message | Creates new stream segment | Creates or continues session |
resume() | Continue interrupted session | Continues existing stream | Must have interrupted state |
execute() is for normal agent interactions:
- Starts a new Temporal workflow (or continues an existing session)
- Creates a fresh stream segment for the response
- Appends the new user message to the session
// New conversation
const handle = await executor.execute(agent, 'Hello', { sessionId: 'session-1' });
// Follow-up in same session (new run, new stream segment)
const handle2 = await executor.execute(agent, 'Tell me more', { sessionId: 'session-1' });resume() is specifically for recovering from interrupts or HITL suspensions:
- Starts a NEW Temporal workflow instance with the
__resume-<runId>suffix (the prior workflow has already exited at the interrupt or HITL boundary) - Maintains stream continuity for clients reconnecting (the new workflow emits a
run_resumedchunk so the SSE client can dedupe) - Session must be in
interrupted,paused, or HITL-suspended status (pendingClientToolCallsmap non-empty)
// After an interrupt...
const state = await stateStore.loadState('session-1');
if (state?.status === 'interrupted') {
// Resume starts a new workflow that continues the same stream
const handle = await executor.resume(agent, 'session-1');
}Resume conventions (post-A.2)
The Temporal runtime exits the workflow on every HITL boundary (client-executed tool, approval gate, sub-agent suspension). executor.resume(sessionId) starts a NEW workflow instance with a distinctive workflow ID:
${workflowIdPrefix}__${agentType}__${sessionId}__resume-${runId}runIdis the id the call minted for its run, so the id is unique per entry and the caller writes no counter before it knows whether it wins (it used to be__resume-<N>fromincrementResumeCount()).- Single-dash separator (
__resume-<runId>), distinct fromretry()'s__retry__<runId>. A sub-agent re-spawned by its parent workflow still uses__resume-<n>(nfrom the child'sresumeCount+ 1), and a companion continuation…__persistent__<childSessionId>__continue__<toolCallId>. getHandle()finds the workflow that owns the session's current run from the run's owner token, never by rebuilding an id.ALLOW_DUPLICATEis still passed (harmless with a per-run id).
The new workflow's mode='resume' branch calls activities.applyResultsAndReload, which:
- Drains submitted client-tool results into the message log.
- Fires
onMessage+afterToolhooks for each drained entry. - Synthesizes
client_tool_timeoutresults for expired deadlines. - Drains completed sub-agent children via
recordSubSessionResult. - Returns
{ state, drainedCount, hasMoreSuspended, childrenToRespawn }so the workflow body can decide whether to continue or re-suspend.
A partial client-tool batch is not resumable (R44, CP-86). resume() in continue / with_message mode on a session whose client-tool batch is still partial (some awaited call has no submitted result) is refused with a retryable state_not_resumable before any write or workflow start. Submit every result of the batch first, or interrupt(), or submit error results for the calls you want to abandon. A from_checkpoint rewind is not refused: its new run re-suspends at once on any restored call that is still unanswered. When a resumed run re-suspends without a step (hasMoreSuspended, for example on children that are still suspended), it writes its suspended_* run status, fires onAgentSuspended and pauses the stream, so the session reads paused again (CP-86 and CP-65 are fixed by this release; see the conformance findings).
AgentResult.status for HITL agents resolves with one of:
'suspended_client_tool'— at least one client-executed tool is awaiting submission.'suspended_awaiting_children'— at least one sub-agent child is in'suspended_*'and the parent is awaiting it.'suspended_step_partial'— mixed phase-1 step where some tools completed and others suspended.
result.suspended carries the routing info ({ toolCallIds?, children?, stepId? }) needed to drive the next submitToolResult + resume cycle.
A suspension now pauses the root session's stream after the suspended run status and the internal suspension_marker, so a reader sees run_paused (the AI SDK's data-run-paused) as on JS (it used to stay active, RM-56). A child session writes into its parent's stream and never pauses it. The finding record stays open until its conformance cell lands; see RM-56 in the conformance findings.
abort() mid-step (WA MR2 F4-C). An abort() while a step is in flight cancels the workflow before the step commits. The workflow's terminal runs in a non-cancellable scope, so the cancelled workflow still commits the abort's own K5 (failed / framework_cancelled, resolved from the stop record). That terminal marks the uncommitted step step_discarded (Ruling 66) before it fails the stream. interrupt() is not affected: it commits the in-flight step first. Both are fixed in code; the CP-22 and RM-59 records in the conformance findings flip when the conformance owner re-runs their cells.
Resume modes. The mode rides the workflow input, and the resume workflow's entry performs it inside its run-start commit (previously every mode behaved like continue, CP-31):
from_checkpoint— the checkpoint is resolved before any write (another session's id throwsstate_checkpoint_foreign, an unknown idstate_checkpoint_not_found, both before any workflow starts). The rewind is the run-start commit'struncateTo: the message log is truncated to the checkpoint'smessageCountatomically with the new run, its state restored, and each discarded pending client-tool call gets an abortedtool_end. Anactivesession whose run is still live-owned (no pending client-tool or approval calls) is refused withAgentAlreadyRunningError, and concurrent calls have one winner. Acompletedorfailedsession is rewound too and a new workflow starts from the checkpoint (C7, as on every runtime); onlycontinuetakes the terminal short-circuit, which returns the old terminal result. Awith_messageover acompletedorfailedsession is refused withAgentNotResumableError(state_not_resumable) before any write, as on every other runtime:retry()a failed session, or send the next turn withexecute(). That includes a session an abort stop ended (failed/framework_cancelled).with_message— the message is the run-start commit's input rows, so the first model call sees it, and a call that loses writes nothing.
submitToolResult({ sessionId, toolCallId, result | error }) is a durable write only — no Temporal signal is sent. The workflow has already exited at the HITL boundary; consumers drive the next loop iteration via executor.resume() (or via the framework's handleChatStream plumbing on the server side).
A submit for a call of a stopped run stores nothing (WA MR2 §7.1): the submit reads the stop of the call's run on its owner session first, and the answer is { status: 'unknown_tool_call', reason: 'run_stopped' }. A stop that is recorded but not yet resolved is settled through the stop path (stopRun with the record's own kind and reason):
- the run's workflow has exited (the usual case: a Temporal workflow exits at a HITL boundary), so the stop's terminal commits before the answer (
interrupted, orfailed/framework_cancelled); - the run's workflow is still open (for example, still in its suspension tail), so it is only signalled (
INTERRUPT_SIGNAL_NAME; an abort also cancels it), never terminated. The answer is stillrun_stopped, but it arrives before the terminal lands: the workflow ends the run by the stop when it reads the record, and a session read right after the answer can still show the suspended run.
A stop that already ended the run answers the same while the call is still unanswered in the log.
Agent Registry
Register agents so the worker can instantiate them:
import { AgentRegistry } from '@helix-agents/runtime-temporal';
const registry = new AgentRegistry();
// Register each agent type
registry.register(ResearchAgent); // name: 'researcher'
registry.register(AnalyzerAgent); // name: 'analyzer'
registry.register(SummarizerAgent); // name: 'summarizer'
// The built-in activities resolve agents by type:
// new GenericActivities({
// agentResolver: { resolveAgent: (type) => registry.get(type) }, // static agents only
// stateStore, streamManager, llmAdapter,
// })
// In an activity of your own, look up by type
export async function myAgentActivity(agentType: string) {
const agent = registry.get(agentType); // Returns the agent config
// ...
}AgentRegistry with Factories
The AgentRegistry supports both static registration and factory functions for dynamic agent creation. Factories are useful when agents need runtime dependencies like API keys, database connections, or per-request configuration.
Static vs Factory Registration
import { AgentRegistry } from '@helix-agents/runtime-temporal';
const registry = new AgentRegistry();
// Static registration - for agents with no runtime dependencies
registry.register(ResearchAgent);
registry.register(SummarizerAgent);
// Factory registration - for agents needing runtime dependencies
interface Env {
TAVILY_API_KEY: string;
DATABASE: Database;
}
registry.registerFactory<Env>('dynamic-researcher', (ctx) => {
return createResearchAgent({
apiKey: ctx.env.TAVILY_API_KEY,
database: ctx.env.DATABASE,
userId: ctx.userId,
sessionId: ctx.sessionId,
});
});AgentFactoryContext
Factory functions receive context about the current execution:
interface AgentFactoryContext<TEnv = unknown> {
/** Environment bindings (e.g., API keys, connections) */
env: TEnv;
/** Current session identifier */
sessionId: string;
/** Current run identifier */
runId: string;
/** Optional user identifier */
userId?: string;
}Using resolve() in Activities
The resolve() method handles both static and factory agents. Use it instead of get() when you have factories, for example in an activity of your own:
// src/activities.ts
export async function myAgentStepActivity(
agentType: string,
state: AgentState<unknown, unknown>,
context: { sessionId: string; runId: string; userId?: string }
): Promise<StepResult> {
// resolve() handles both static and factory agents
const agent = registry.resolve(agentType, {
env, // Your environment object
sessionId: context.sessionId,
runId: context.runId,
userId: context.userId,
});
return executeStep(agent, state, llmAdapter, streamManager);
}Error Handling for Unknown Agent Types
An unknown agent type throws core AgentResolutionError.notFound (code: 'framework_agent_not_found', non-retryable, with agentType and availableTypes) from both get() and resolve(). get() returns only static registrations: for a factory-registered type it throws the same code with a message that says to use resolve(). Detect a miss with core isAgentNotFoundError, which matches by name and code (not instanceof). The registry stays synchronous (resolve() returns the config, never a promise), because activities resolve through your synchronous resolveAgent.
import { isAgentNotFoundError } from '@helix-agents/core';
try {
const agent = registry.resolve('unknown-agent', context);
} catch (error) {
if (isAgentNotFoundError(error)) {
console.error(`Unknown agent type: ${error.agentType}`);
console.error(`Available types: ${error.availableTypes?.join(', ')}`);
}
}A miss that escapes an activity still reaches the run as the wrapping ActivityFailure's detail, after the activity retries, not as framework_agent_not_found (DI-04). See the DO drive pipeline upgrade guide.
Registry Methods
| Method | Description |
|---|---|
register(config) | Register a static agent configuration |
registerFactory(type, factory) | Register a factory function |
replace(config) | Overwrite an existing static registration |
resolve(type, context) | Resolve agent (factory-first, then static) |
get(type) | Get a static registration only |
has(type) | Check if agent type is registered |
unregister(type) | Remove an agent registration |
getRegisteredTypes() | Get all registered agent type names |
clear() | Remove all registrations (for testing) |
registry.replace(config)
Overwrites an existing static agent registration with a new config, returning whether a prior registration existed. Throws if a factory is registered under the same name.
const registry = new AgentRegistry();
registry.register(myAgent);
// Later: swap the registered reference (e.g., to attach hooks)
const agentWithHooks = { ...myAgent, hooks: makeHooks() };
const hadPrior = registry.replace(agentWithHooks);
// hadPrior === trueWhen to use: The Temporal runtime resolves agents BY NAME from the registry — workflow inputs only carry the agent name, not the agent reference. Per-call hooks attached to the agent passed into executor.execute(agent, ...) aren't honored unless the registered reference itself carries them. replace() is the canonical way to swap the reference (most commonly in tests that exercise per-test hooks). JS / DBOS runtimes read the agent reference inline at execute() time and don't need this API.
Don't replace factories. If a factory is registered under the same name, replace() throws. Use unregister() followed by registerFactory() to replace a factory.
Sub-Agent Handling
Sub-agents execute as child workflows:
// In workflow
import { executeChild } from '@temporalio/workflow';
// When parent needs to execute sub-agent
const subAgentResult = await executeChild('agentWorkflow', {
args: [
{
agentType: 'analyzer',
sessionId: `${parentSessionId}-sub-${callId}`,
streamId: parentStreamId, // Same stream for unified streaming
message: inputMessage,
parentSessionId,
},
],
workflowId: `agent__analyzer__${subSessionId}`,
taskQueue: 'agent-tasks',
});Benefits of child workflows:
- Independent retry policies
- Separate timeouts
- Can be cancelled independently
- Full workflow history preserved
Remote Sub-Agent Handling
Remote sub-agents are executed via a dedicated executeRemoteSubAgentCall activity, separate from both regular tool calls and local sub-agent child workflows.
The activity provides:
- Deterministic session IDs —
{parentSessionId}-remote-{toolCallId}for idempotent restarts - Crash recovery — On activity retry, checks
transport.getStatus()to avoid re-executing completed agents - Heartbeat-based reconnection — Heartbeats store
lastSequenceso retries can resume the SSE stream mid-flight - Stream proxying — Remote agent chunks flow through the parent's stream
- SubSessionRef tracking — Registers with
remote: { streamId, lastSequence }metadata - Interrupt propagation — Racing pattern propagates interrupts to abort the remote call
- Trace propagation — An entry's
telemetryParent(EntryTraceOptionsonexecute,resumeandretry) is recorded in the workflow input astraceParent(serializedTraceIds). The activity pins its transport to it (coretransportFor), covering the dispatch and the completion fold's reads, so the remote child joins the caller's trace. Every child workflow (local sub-agents and companions) inherits it. No tracer provider is needed, and Temporal emits no spans of its own yet (FU-OTEL-TEMPORAL). The field rides input payloads, which Temporal's determinism check does not compare, so it needs nopatched()guard; a workflow without it behaves as before.
See Remote Agents Guide for setup and configuration.
Persistent Sub-Agent Handling
Persistent sub-agents in the Temporal runtime execute as child workflows managed through companion tools:
- Blocking spawn: Uses
executeChild()to run the child as a Temporal child workflow. The parent workflow waits for the child to complete. After completion, the workflow callsmarkPersistentChildStatusto update the SubSessionRef. - Non-blocking spawn: Starts a child workflow with
parentClosePolicy: 'ABANDON'. The parent continues immediately. - Companion tool execution: All companion tool calls route through a dedicated
executeCompanionToolCallactivity, separate from regular tool execution.
Unlike the JS runtime, the Temporal runtime does not store companion tool results as separate ToolResultMessage entries in the state store. Results flow through the workflow execution context.
Persistent children are tracked via SubSessionRef entries with mode: 'persistent' and are cleaned up when the parent workflow completes.
Re-spawning a completed persistent child continues it on its preserved session (memory retained) rather than recreating it — see Re-consulting a persistent companion (the critic loop).
- Continuation marker. A consult that reopens a
completedchild (completed → active, one version-pinned write, so one consult wins) records a pending-continuation marker in the child session's metadata under__helix.companionContinue. The child's continuation entry clears it once its run commits; a continuation that fails before its run commits rolls the reopen back, so the child is never strandedactive. The key is visible tolistSessionsmetadata filters. The continuation runs in its own child workflow,…__persistent__<childSessionId>__continue__<toolCallId>. - Detached children. A non-blocking child keeps its content in its own session. Its live chunks that reach the parent's stream after the parent's run ended or paused that stream are dropped instead of failing the child; its content stays in its own session and reaches the parent through the snapshot and the next turn's notification.
Known companion gaps on Temporal (CP-41, CP-72, CP-73, CP-105)
companion__sendMessageto a running child appends the message to the child's live run and records no stop (WA MR2); the child's workflow loads the history at its next step. The CP-41 record stays open until its cells are re-run on Temporal.- Two
sendMessagecalls to one interrupted child in the same step compute the same child workflow id. If the two starts overlap, the second fails with only a log line (CP-72). - When a child ends
interrupted, its parent-side ref staysrunning, solistChildren/getChildStatusreportrunning(CP-73). An untimedcompanion__waitForResultno longer hangs on it: without a liveness probe it also checks the child's session and that it has no live run, and returnsinterruptedafter three polls. Known bug: a same-stepsendMessageresume whose child takes more than about 2 s to reach its run-start commit can be reportedinterrupted(FU-TEMPORAL-WAITFORRESULT-RESUME-RACE). - A
sendMessageresume of an interrupted child writes the parent's refrunningonly in the child's own run-start commit (companionResume,StartRun.resumedCompanionRef; an entry withcompanionResumeand noparentSessionIdfailsvalidation_errorbefore any write), and a resume the worker's adapter cannot finish is refused in the parent's companion activity as a tool error (CP-103, fixed). But a resume whose child loses its run-start to another entry, or whose parent ref is gone, still returnsdelivered: true(CP-105). In each of these cases the resume writes no ref, so the ref keeps its prior status: the stalerunningof CP-73, since Temporal never recordsinterruptedon a ref.
See the conformance findings.
Activity Configuration
The packaged agentWorkflow proxies its activities itself, so their timeouts and retry policy are fixed; neither the executor nor the worker configures them:
- Every activity except the terminal finalization:
startToCloseTimeout: '5 minutes', retryinitialInterval: '1 second',backoffCoefficient: 2,maximumInterval: '10 seconds',maximumAttempts: 3. - The terminal finalization (
finalizeTerminalRun: the run's terminal status and the stream end, after the terminal commit) runs on its own budget: unlimited attempts within 3 minutes (scheduleToCloseTimeout), each attempt at most 1 minute. - A remote sub-agent call (
executeRemoteSubAgentCall) also gets a 30-second heartbeat timeout and a start-to-close timeout sized from the call, with the same 3-attempt retry policy.
The 3-attempt budget is what bounds a forced call's retryable provider errors (see Completion strategies).
Error Classification
When LLM calls fail during Temporal activities, errors are classified using the same HelixError system as other runtimes. The classification has to cross the Temporal serialization boundary:
- Activity --
runLLMStepclassifies the failure. The adapter'sonErroris held (coreholdLLMErrorReport) until the step's outcome is known: anerrorchunk (withcodeandrecoverable) is written only for a terminal failure, never for an attempt that an activity retry or a recoverable error result will re-run, because a client stops reading at anerrorevent. The held report is written as the activity's last step, after the attempt's usage recording,afterLLMCall/onMessagehooks and other writes, so an attempt retried because one of them threw writes noerrorchunk. - Workflow -- A terminal failure ends the run through the terminal order: the terminal commit with
failedand the error, the run status, then the failed stream. The activity computes the step's classifiedErrorDetailwhile the real error exists and returns it (terminal.errorDetail), so an ordinary terminal LLM error keeps itscode,categoryandretryableonAgentResult.errorDetail, the persistedSessionState.errorDetailand the erroronAgentFailreceives (getErrorDetail(error)). Other activity failures still fall back to the wrappingActivityFailure's detail (DI-04). - Retries -- A retried activity streams its step again under a new
attemptId; the superseded attempt's content is dropped by readers.
See Error Handling Guide for the complete error pipeline.
A blocked, refused or truncated final reply completes the run (DI-26)
For an agent without outputSchema, the workflow never classifies a text step's stop reason. A final step that stops with content_filter, refusal, max_tokens, error or unknown ends the run completed with no error, where JS and both Cloudflare runtimes fail it. Observed live for content_filter and max_tokens; the other reasons follow from source. Agents with an outputSchema are not affected. See the error handling guide and DI-26 in the conformance findings.
Completion strategies
Temporal runs completion strategies like the other runtimes, so an outputSchema agent finishes on Claude Opus 5.5, Sonnet 5.5, Fable 5.1 and Mythos 5.1 (through native_output). The worker's LLM adapter decides; the client executor has none.
- Per call, in the
runLLMStepactivity. The activity resolves the model's capabilities on every LLM call and, on a forced call, builds the directive for the strategy it selects (forced_toolornative_output). Temporal never runsllmConfigOverride, so the agent's configured model is the forced call's model. Anative_outputcall streams notext_delta; hook payloads (completionStrategy,capabilityProfile) and theattempt_startedchunk (strategy,capabilityProfile) record what was applied. The activity also decides the forced call's outcome and returns it to the workflow, which applies it. - Early refusal. An
outputSchemaagent whose model has no strategy, or whose completion schema the selectednative_outputmodel cannot take, is refused in theinitializeAgentState/startRunCommitactivity, before its run-start commit:execute(),resume()andretry()throw a non-retryableHelixError(framework_completion_strategy_unavailable/framework_completion_schema_unsupported). A brand-new session (one with no run yet) is markedfailedwith the typed detail and its stream is failed (root sessions); an existing session is left as it was. A sub-agent or companion is refused at its own start, so the parent sees the child fail (a refused companion re-consult reportsdelivered: falsewith the refusal'scodeanderrorDetail). The check resolves the agent through youragentResolver. Only a registry miss (coreAgentResolutionErrorwithcode: 'framework_agent_not_found', whichAgentRegistry.resolve/getthrow; matched withisAgentNotFoundError) skips it: the entry commits its run and fails at its first resolve. Any other resolver error, such as an agent factory that throws, fails theinitializeAgentState/startRunCommitactivity (or, for a companionsendMessageresume, the companion activity) before anything is committed or any child starts. A customresolveAgentshould throwAgentResolutionError.notFound(type, availableTypes)for an unknown type. - Provider errors on a forced call. A retryable or unclassified error is rethrown so the activity retry policy (
maximumAttempts: 3) repeats the same forced attempt without consuming one; when the retries run out, the run fails with the provider's detail and its forced phase is closedfailedin the terminal commit. A non-retryable error fails the run and the phase at once.afterLLMCallfires once per attempt, so it can fire more than once for one forced attempt. The hook'sresultdoes not carryonErrorFailurehere: the held error report is written after the hook, so a failed error-chunk write is only known afterwards (logged, and set on the returned step result). - Prompt-prefix check. On a model whose thinking is bound to history, the activity fingerprints the request and refuses a changed system prompt or tool set with
framework_prompt_prefix_changed(Preserved thinking) before the attempt'sstep_start, so a refused attempt writes no attempt chunk. The assistant message records the fingerprint (metadata.helixPrefix). - Hook order.
beforeLLMCallfires before the attempt'sstep_start(it used to fire after it), so a chunk a hook emits withctx.emitnow precedes thatstep_start. - Terminal LLM errors keep their classified
errorDetail(code,category,retryable) on the result, the persisted session andonAgentFail(DI-24, see Error Classification).
Deploying
Every workflow-side decision is gated by wf.patched('helix-completion-strategy-v1'). A workflow started before the upgrade that reaches a gated point while running on an upgraded worker records the patch marker and takes the new path from there on; a replay of its earlier history keeps the old path, so recorded workflows replay deterministically. The exposure is the rollout overlap, while old and new workers both poll the task queue. A workflow task processed by an old worker that receives a runLLMStep result produced by a new worker classifies it the old way: a forced call's non-retryable provider error is then written as an error chunk by the new activity but counted as a miss by the old workflow code, so such a run may show an extra error chunk and can end framework_forced_completion_exhausted. An old worker that picks up a workflow task of a workflow that already took the new path fails that task with a nondeterminism error, which Temporal retries until a new worker takes it. A new workflow's runLLMStep executed by an old worker gets the legacy request, which on a 5.5-generation model can end framework_forced_completion_exhausted as before the upgrade. Once every worker runs this version none of this applies. (An upgraded worker that receives a result from an older activity also classifies it the legacy way.) Keep the patch until every workflow started before the upgrade has finished (FU-TEMPORAL-COMPLETION-PATCH-DEPRECATE). The one workflow-side change that is not gated is an activity argument: a retry() workflow passes retryEntry: true to initializeAgentState, so the retried run keeps the forced phase startRunCommit restored. Replay is unaffected, because Temporal matches commands on activity type and id, never on arguments. During the overlap it can mix versions: a retry whose startRunCommit ran on an old worker restores the target's raw phase (without the new correction for a forced step's own transition commit), and a new worker's initializeAgentState then keeps it, so a retry after an exhausted forced phase re-forces the spent attempt once (and a budget-0 terminal_text / max_tokens retry restores no phase instead of failed); when the hook half runs on an old worker, or the workflow task runs old code (which passes no argument), the phase is reset as before the upgrade and the retried run starts with a work call.
Crash Recovery
Temporal provides automatic crash recovery:
graph TB
W1["Worker 1 starts workflow"]
W1 --> S1["Step 1 completes, state saved"]
S1 --> S2["Step 2 completes, state saved"]
S2 --> Crash["Worker 1 crashes"]
Crash --> Detect["Temporal detects failure"]
Detect --> Resched["Workflow task rescheduled"]
Resched --> W2["Worker 2 picks up"]
W2 --> Replay["Replays history (deterministic)"]
Replay --> Continue["Continues from Step 3"]
Continue --> Complete["Completes normally"]Key points:
- Workflow code must be deterministic
- State is reconstructed from event history
- Activities are not re-executed (results cached)
Interrupt and Resume
The Temporal runtime supports soft interruption with the ability to resume later:
// Interrupt the agent (soft stop): observed before its next step's LLM call
await handle.interrupt('user_requested');
// The run ends 'interrupted' once the workflow has observed it
const interrupted = await handle.result();
console.log(interrupted.status); // 'interrupted'
// Later, resume execution
const { canResume } = await handle.canResume();
if (canResume) {
const newHandle = await handle.resume();
const result = await newHandle.result();
}Handles do not pass resume options on (CP-31)
An interrupted run ends the session's stream. The resume's run-start winner now claims the stream and makes it live again before it writes, as every entry does (it used to leave the stream ended, so the resumed turn could not write, CP-76; that record stays open until its conformance cell lands). executor.resume() honours mode and message (see Resume modes above), but no handle passes them on: the execute() handle's and the factory handle's resume() throw, and a getHandle() handle calls executor.resume(agent, sessionId) without its options (CP-31). Call executor.resume(agent, sessionId, options) directly. See the conformance findings.
Workflow Setup (v7)
The v7 stateless workflow exits at every HITL boundary and the executor spawns a fresh __resume-<runId> workflow on each resume. Its INTERRUPT_SIGNAL_NAME handler only wakes the running execution; the run's durable stop record in the state store is the canonical channel (see Interrupts and sub-agents). Register agentWorkflow directly, or re-export it under a stable name your TemporalAgentExecutor.workflowName expects:
import { agentWorkflow as v7AgentWorkflow } from '@helix-agents/runtime-temporal/workflow';
import type { AgentWorkflowInput, AgentWorkflowResult } from '@helix-agents/runtime-temporal';
// Re-export under your workflow name. The imported `agentWorkflow` sets up
// `proxyActivities<GenericActivities>`, the INTERRUPT_SIGNAL_NAME handler,
// and child-workflow dispatch internally. The wrapper exists only to
// register the function under a stable name.
export async function agentWorkflow(input: AgentWorkflowInput): Promise<AgentWorkflowResult> {
return v7AgentWorkflow(input);
}Removed v6 signals
v7's stateless suspension model removed four Temporal signals: submitToolResult, childSuspended, childWoke, runResumed. Code that called handle.signal('submitToolResult', ...) or used the RUN_RESUMED_SIGNAL_NAME constant must migrate to the v7 stateless contract — submission writes to durable state, and executor.resume() spawns a fresh __resume-<runId> workflow that drains submissions via the applyResultsAndReload activity. The two-arg runAgentWorkflow(input, activities, options) signature is also gone; the v7 export is agentWorkflow(input). See the v6 → v7 upgrade guide. INTERRUPT_SIGNAL_NAME ('agent_interrupt', exported from @helix-agents/runtime-temporal/workflow) is the one signal the workflow still handles: the executor's handles send it, carrying the recorded StopRequest.
Interrupts and sub-agents
An interrupt stops the run it is sent to, at a step boundary:
- Request - Every handle's
interrupt()(theexecute(),resume(),retry()andgetHandle()handles) stops the handle's own run through corestopRun(WA MR2): the stop is recorded on the run, then, while the run's workflow execution is open, signalled (INTERRUPT_SIGNAL_NAME, carrying theStopRequest). A run whose workflow is closed (a suspended run) is ended by the call itself. A run that already ended resolves the call (logged). The signal only wakes the workflow; the run's durable stop record reaches it from any process, and the workflow reads it with thereadStopActivityactivity. - Observation - The workflow reads the record at every main-loop iteration, before that step's LLM call, and on a resume, right after the drain of submitted results (
applyResultsAndReload), before it decides whether the run stays suspended. It is never checked while a step's LLM call, tools or sub-agent children are in flight, so they finish first. Every live-run commit is guarded (refuseIfStopped), so a stop that lands before a commit ends the run by the stop instead. - Stop - The run settles pending client-tool calls with an aborted
tool_end, commits its terminal (interrupted; an abort:failedwithframework_cancelled), then writesrun_interruptedand returns that committed outcome. The terminal is announced only after it commits (WA MR2 F4-C S3).
abort() also cancels the workflow, and resolves once the stop is durable (recorded, the workflow signalled and cancelled; F4-C m4). It never waits for the workflow to close. The cancelled workflow commits its own terminal (its terminal runs non-cancellable), and result() reports it. After stopRun answers, abort() makes one bounded owner check: a run still live under a workflow that is ALREADY closed at that check is settled by the call itself. A workflow that finishes closing just after the check (for example, one completing its suspension when the cancel lands) is not: a handle reading its result() settles it, else it waits for FU-TEMPORAL-STOP-SETTLE-CLOSED-OWNER. A failure of that check is logged and never rejects abort() (the stop is already durable). agent-server's stop routes, which call the handle, answer at once too.
A terminal that did not commit (F4-C S3). When a run's K5 fails for any reason other than a lost claim, the workflow retries it once in its top-level catch. If that also fails, the workflow fails with the typed temporal_terminal_not_committed failure; it never returns a synthesized interrupted / failed result over a session that is still active. The handle's result() then reconciles the run from the store, because the owner is now closed:
- a run that already ended, or whose own K5 landed, reports that outcome;
- a recorded stop gets the stop's terminal;
- otherwise the run fails with the failure it was ending with (core
failCommittedRun).
If that settle does not land either, result() rejects with its error and caches nothing, so a later result() retries.
A handle whose workflow ended suspended although its run's stop was recorded settles the run by that stop and reports the stopped outcome. Two cases are still open:
- a stopped run whose workflow closed with no handle reading its result stays live until the next entry settles it (FU-TEMPORAL-STOP-SETTLE-CLOSED-OWNER);
- a stop recorded after a suspension commit, with no handle reading, is settled only by the next continuation (FU-TEMPORAL-STOP-AFTER-SUSPENSION-COMMIT).
Child workflows are not signaled. On an interrupt, a sub-agent child the step is waiting on runs to its own end before the parent sees the stop. On an abort, the parent workflow's cancellation reaches the children it is waiting on (Temporal's child cancellation): each child is cancelled with it and commits its own terminal, failed with the cancellation's message and no framework_cancelled code, because the child has no stop record of its own (FU-TEMPORAL-CHILD-CANCEL-UNTYPED-FAILURE). A non-blocking persistent companion (started with ParentClosePolicy.ABANDON) keeps running either way. Temporal does not record 'interrupted' on a companion's SubSessionRef (Sub-Agents).
See Interrupt and Resume for complete documentation including resume modes and error handling.
Stream Resumption
The Temporal runtime supports stream resumption for client disconnections and page refreshes with the same @helix-agents/ai-sdk handlers as the other runtimes: buildSnapshot for the snapshot route and handleChatStream for the chat route.
Setting Up Stream Resumption
import { buildSnapshot, handleChatStream } from '@helix-agents/ai-sdk';
const deps = { executor, stateStore, streamManager, agent: myAgent }; // TemporalAgentExecutor
// GET /api/chat/:sessionId/snapshot
const snapshot = await buildSnapshot(deps, { sessionId });
// POST /api/chat/:sessionId (send or resume)
return handleChatStream(deps, { sessionId, messages, request });Frontend Integration
Seed the chat from the snapshot and resume only while the run is live, with snapshotResumeOptions:
import { snapshotResumeOptions } from '@helix-agents/ai-sdk/client';
const opts = snapshotResumeOptions(snapshot);
// opts.messages: the seed (without the in-flight message when resuming)
// opts.resume: true only for status 'active' / 'paused'
// opts.resumeFromSequence / opts.existingMessageId / opts.replayPrefix: the reconnect targetA snapshot is taken from one checkpoint: its messages are exactly the rows committed up to its streamSequence cursor, and a resume streams everything after that cursor (the server replays the in-flight message's committed prefix first). There is no partial-content merge and no content-replay option any more. See Resumable Streams and the upgrade guide for the full client wiring.
Key Considerations
- Durable stream manager - Stream resumption relies on a stream manager that stores chunks with sequence numbers (for example
RedisStreamManager) - Chunk retention - Configure appropriate TTL for stream chunks based on expected session duration
- Multi-worker - Works across multiple workers since stream state is in the shared store
- Retried activities - A retried activity re-streams its step under a new
attemptId; resumed readers drop the superseded attempt
Concurrency Protection
The run-start commit is the single gate (see Run Start and Ownership); workflow ids add a second guard:
- Every
execute()turn of a session runs under the session's base workflow id, and Temporal refuses a duplicate while one is running - Concurrent
execute()calls result in workflow collision errors (AgentAlreadyRunningError) resume()/retry()run under per-run ids (__resume-<runId>,__retry__<runId>); for them the run-start commit alone picks the winner, and a loser writes nothing
Those collision errors are not only for concurrent calls: an execute() for the next turn made right after the previous turn reads completed, but before its workflow has closed, collides too (CP-84, see Multi-Turn Conversations).
resume() against a live run is refused (CP-83)
A session that reads active with no pending calls and whose current run's workflow is still running is a live run: resume() refuses it in every mode with AgentAlreadyRunningError before any write, the rule DBOS enforces. A session whose run's workflow closed (a crashed or terminated execution) is not refused: the resume's run-start commit supersedes the stale run. In earlier releases each such resume started a new __resume-N workflow beside the live one, and the live run ended failed with its answer lost. See CP-83 in the conformance findings.
retry() Method
Retry failed agents in Temporal:
const result = await handle.result();
if (result.status === 'failed') {
const retryHandle = await executor.retry(agent, sessionId);
// or
const retryHandle = await handle.retry();
}The retry creates a new workflow execution (…__retry__<runId>) whose entry is one run-start commit: it rewinds to the retry target and appends the input atomically. The target is core planRetry, the same on every runtime: by default the failed run's last committed step (pass message, or the retry throws validation_error), or, when the run failed on its first step, its run-start commit (the triggering message is re-run, followed by message if given). The terminal checkpoint a failure writes is never the target. retry({ checkpointId }) resolves the checkpoint before any write: an unknown id throws state_checkpoint_not_found and another session's id throws state_checkpoint_foreign, with the session left untouched. See JS runtime: retry() for the full rules.
The retried run restores the target's customState, stepCount and forced-completion phase, as on runtime-js. A retry into a pending forced phase continues the restored attempt: the retried run's first call is the forced call, resolved for the adapter the worker has now. A default retry whose target is the forced step that recorded the model's miss on the phase's last attempt (under either strategy: a text, invalid or protocol-violating answer to a __finish__ forced call, or a native_output call's prose or schema-invalid answer) restores the phase as that step's own forced transition left it, so that attempt is not re-forced. Only a miss the forced step recorded is spared: every other failure inside a forced phase restores it pending and re-forces the attempt, as on runtime-js. For example, a finishWith failure (the default target is the phase-2 append commit, which holds the phase pending), or a provider error, retryable or not (the forced step records no row, so the target is the commit before it, also pending) (FU-COMPLETION-RETRY-AFTER-FINISHWITH-EXHAUSTION). A maxCompletionRetries: 0 run that failed at a terminal_text or max_tokens trigger restores a failed phase, one that failed at a max_steps trigger restores none, as on runtime-js. (Before @helix-agents/runtime-temporal 4.0 the retry's entry reset the phase, so a pending phase was dropped and the retried run's first call was a work call.) A retry() against an outputSchema agent whose model has no completion strategy is refused before its run-start commit (see Completion strategies).
Branching (execute(agent, input, { branch })) now honours branch.checkpointId and branch.messageIndex (CP-33): the new session gets exactly the checkpoint's first messageCount messages and its customState, or the first messageIndex messages. A foreign or unknown branch.checkpointId is rejected by execute() itself, before startWorkflow.
Retrying to a finishWith step's checkpoint deletes the call's result (RM-52)
The workflow commits a step (its step checkpoint) before runPhase2FinishWith runs, and the finishWith result is appended afterwards by its own append commit. So the step checkpoint of a step that called a finishWith tool ends on the tool_use without its result. It is no longer the turn's latest checkpoint (the append and terminal commits follow it), and a default retry() never targets it, but a retry({ checkpointId }) that names it truncates the transcript there: the call's real result is deleted and its tool_use is left unpaired (observed on the memory store before this release). Do not retry to the checkpoint of a finishWith step. See RM-52 in the conformance findings.
The handle executor.resume() returns now settles result() on the resumed workflow's own result, as the execute() handle does, and the terminal commit (with failed) is written before the workflow returns. It used to settle on the first error chunk, before the failure was stored, so an immediate retry() could throw (CP-80; the record stays open until its conformance cell lands; see the conformance findings).
Method Comparison
| Method | Purpose | Stream Behavior | Valid From Status |
|---|---|---|---|
execute() | New/continue conversation | Resets stream | Any except running |
resume() | Continue after interrupt | Preserves stream | interrupted, paused (incl. HITL suspension); any status with from_checkpoint once its run is not live |
retry() | Recover from failure | Resets to its target | failed |
Determinism Requirements
The packaged agentWorkflow is deterministic: all of its I/O runs in activities. Workflow code of your own must be deterministic too:
// BAD - Non-deterministic
export async function myWorkflow(input) {
const timestamp = Date.now(); // Different on replay!
const random = Math.random(); // Different on replay!
const uuid = crypto.randomUUID(); // Different on replay!
}
// GOOD - Use Temporal APIs
import { sleep, uuid4, workflowInfo } from '@temporalio/workflow';
export async function myWorkflow(input) {
const info = workflowInfo();
const timestamp = info.startTime; // Deterministic
const id = uuid4(); // Deterministic (seeded)
await sleep('5 seconds'); // Deterministic timer
}Move non-deterministic operations to activities:
- LLM API calls
- Database queries
- External API calls
- Random number generation
Observability
Temporal Web UI
Access at http://localhost:8233 (local) or via Temporal Cloud.
View:
- Workflow history and events
- Activity execution details
- Pending/failed workflows
- Search by workflow ID or type
Workflow Queries
agentWorkflow registers no query handlers: a run's progress lives in the session store, not in workflow state. Read it with handle.getState() (status, step count, custom state) or follow the stream.
Production Deployment
Worker Scaling
Run multiple workers for throughput:
# Scale horizontally
docker-compose scale worker=5Workers pull from the same task queue - Temporal handles distribution.
Temporal Cloud
For production, use Temporal Cloud:
import { Connection, Client } from '@temporalio/client';
const connection = await Connection.connect({
address: 'your-namespace.tmprl.cloud:7233',
tls: {
clientCertPair: {
crt: fs.readFileSync('client.pem'),
key: fs.readFileSync('client.key'),
},
},
});Monitoring
Set up metrics:
import { Runtime } from '@temporalio/worker';
Runtime.install({
telemetryOptions: {
metrics: {
prometheus: { bindAddress: '0.0.0.0:9464' },
},
},
});Limitations
Workspaces not supported
The Temporal runtime does NOT wire workspace providers. execute() / resume() / retry() of an agent that declares a workspace fail fast at run start with an error of the form:
Temporal runtime does not support workspaces (yet — design-reserved).
Agent '<agentType>' declares a workspace.
Use one of: JS runtime, Cloudflare Durable Object runtime via @helix-agents/agent-server.The check (assertRuntimeSupportsWorkspaces, shared with the Cloudflare Workflows runtime) runs synchronously at the top of each entry point, before any state load or workflow start, so no side effects occur when it rejects. describeCapabilities() returns an empty workspaceProviderKinds array so AgentServer's executor-side check catches a workspace-declaring agent at construction time too.
If your agent needs files, shell, code, or snapshots, run it on the JS runtime or the Cloudflare DO runtime via @helix-agents/agent-server (both DO wire workspaces — see the workspaces overview for provider options). Workspace support on Temporal is design-reserved for a future plan.
stopWhen is ignored and a maxSteps cut-off completes
stopWhen is never evaluated on Temporal (DI-29)
The workflow calls the stop check (shouldStopExecution) with maxSteps only, so an agent's stopWhen predicate never runs: after it would have fired, the agent keeps calling the LLM and running tools until the model stops on its own or maxSteps is reached (50 when the agent sets none; a sub-agent or companion child workflow is not capped, FU-TEMPORAL-CHILD-MAX-STEPS). Observed live (a stopWhen: () => true agent made 3 LLM calls where JS made 1). JS, the Cloudflare DO and Cloudflare Workflows honour it. Use a finishWith tool or an outputSchema to end the loop. See the stopWhen guide and DI-29 in the conformance findings.
A maxSteps cut-off completes on Temporal (DI-25)
An agent without outputSchema that runs out of maxSteps mid-tool-loop ends completed with no output and no error code here. JS, the Cloudflare DO and Cloudflare Workflows fail it with framework_max_steps_exhausted and let retry() continue it; on Temporal retry() refuses the session because it is not failed. Wiring core resolveStopDecision and the run's completionReason into the workflow is FU-CFD1C-DI29-DI25-TEMPORAL (CF-D1 MR 2). See When maxSteps runs out.
Memory auto-injection/extraction not supported
The Temporal runtime does not wire any MemoryManager logic. The memory: config field on an agent definition is accepted at the type level but is a silent no-op — no memory injection into prompts and no extraction after tool calls. Agents that need live memory recall or storage must run on the JS or Cloudflare runtimes.
LLMConfig.cache strategies are applied on the Temporal runtime (as on the JS, Cloudflare, and DBOS runtimes).
Higher Latency
Each activity invocation adds network overhead. Batch operations when possible.
Determinism Constraints
Workflow code restrictions can be challenging. Move all I/O to activities.
Infrastructure Overhead
Requires running Temporal server and workers alongside your application.
Learning Curve
Temporal concepts (workflows, activities, replay, determinism) require understanding.
Best Practices
1. Keep Workflows Thin
agentWorkflow only orchestrates; every read and write is an activity. Do the same in workflows of your own:
// Workflow just coordinates
export async function myWorkflow(input) {
const state = await loadState(input.sessionId);
const result = await executeStep(state); // Activity does the work
await saveState(state);
}2. Appropriate Timeouts
agentWorkflow sets its own activity timeouts (see Activity Configuration). In a workflow of your own (for example one that starts agentWorkflow as a child and does other work around it), give your activities realistic timeouts:
const { fetchCustomerRecord } = proxyActivities<typeof myActivities>({
startToCloseTimeout: '5 minutes',
});3. Keep Tools Within the Activity Budget
Each server tool runs in an executeToolActivity activity under agentWorkflow's fixed policy: a 5-minute start-to-close timeout and 3 attempts, with no heartbeat timeout (see Activity Configuration). A tool that can run longer should start the work elsewhere (for example a workflow of your own) and return a handle to it. In activities of your own that set a heartbeatTimeout, heartbeat while they work:
import { heartbeat } from '@temporalio/activity';
export async function processItems(items: Item[]): Promise<void> {
for (const item of items) {
await processItem(item);
heartbeat(); // Report progress
}
}4. Histories Stay Per Run
agentWorkflow does not continue-as-new. Every entry (execute() turn, resume(), retry()) starts a new workflow execution, so one execution's history holds one run's steps. A long-lived workflow of your own should reset its history with continueAsNew:
import { continueAsNew, workflowInfo } from '@temporalio/workflow';
export async function myLongWorkflow(input) {
if (workflowInfo().historyLength > 10000) {
await continueAsNew<typeof myLongWorkflow>(input);
}
// ...
}Next Steps
- Interrupt and Resume - Complete interrupt/resume documentation
- JavaScript Runtime - Simpler option for development
- Cloudflare Runtime - Edge deployment alternative
- Storage: Redis - Recommended store for Temporal