Skip to content

A stream chunk over the store's limit is typed, an oversized end part is omitted, and a model's oversized delta is split (RM-90) ​

A stream manager stores each chunk as one value. On the Cloudflare Durable Objects that value is one SQL row, which Cloudflare caps at 2 MB. Before this release, a chunk larger than that failed inside the database:

  • On the agent DO: SQLITE_TOOBIG, after the chunk's sequence was already taken (a gap). The run failed untyped.
  • On the CF Workflows binding-side StreamServer: a 500 "undefined", which the binding turned into a retryable connection error. So the workflow retried the LLM step and called the model again, each time.

Now every stream manager declares its limit, measures the value it binds and refuses an oversized chunk typed, before it allocates a sequence. An end is never refused: an oversized part of its terminal payload is omitted and named, and remote consumers recover it from the run. A model that emits a whole reply as one delta has that delta split at the producer, so it still streams and completes.

It is a minor release of @helix-agents/core, @helix-agents/store-memory, @helix-agents/store-redis, @helix-agents/store-cloudflare, @helix-agents/runtime-js, @helix-agents/runtime-cloudflare, @helix-agents/runtime-temporal, @helix-agents/runtime-dbos and @helix-agents/agent-server (a patch of @helix-agents/store-postgres: a type annotation).

The refusal ​

A chunk whose stored JSON is over StreamManager.limits.maxChunkBytes is refused with core's MA-31 error, StateSizeExceededError (code: 'state_size_exceeded', retryable: false), with a new operation, stream_chunk. It is refused before its sequence is allocated, so the stream has no gap and nothing is broadcast. Over HTTP it is the same 413 as MA-31's (cause: 'stream_chunk'), and every Helix client revives it typed.

STATE_SIZE_OPERATIONS / StateSizeOperation now include stream_chunk. If you switch over operation exhaustively, handle it. The state-store helpers (limitFor, BoundSizeWitness) take the new, narrower StoreSizeOperation (STORE_SIZE_OPERATIONS: stage | step | state | append). A store that annotated a variable as StateSizeOperation to pass to them should use StoreSizeOperation.

An oversized end part is omitted, never refused ​

An end is never refused: that would leave the stream active behind a finished run, and every reader would wait forever. Instead, each part of the end's terminal payload (the final output, state or usage) whose JSON is over the limit is not stored. The stream ends, the manager warns core.stream.end_parts_omitted, and the end names the part:

  • StreamMetadata.endOmitted (getStreamInfo()), and the CF DO's /snapshot streamEndOmitted;
  • the wire end frame and TransportEvent end: a new optional omitted (['output'], …). It is additive: a reader that ignores it sees an end without that part, as before.

Remote consumers recover an omitted part from the child run's own record. Every Helix runtime does it through core processRemoteSubAgentCompletion (now with an omitted in its end source), and so does a remote sub-agent tool's execute():

  • an omitted output is read from getStatus(); a failed read, or a status without an output, fails the call typed (reason: 'output-unavailable', or RemoteAgentFailedError in execute()), never a null output;
  • an omitted state / usage takes the existing getSnapshot() / getUsage() fallback.

If you consume TransportEvents yourself (a bring-your-own loop calling processRemoteSubAgentCompletion), pass the end's omitted in its end source. executeRemoteSubAgentDispatch's completed result carries it as omitted.

With the default limits an oversized final output rarely reaches the stream: it is also stored in the session state, which MA-31's maxStateBytes refuses typed at the terminal commit first.

The limits ​

ManagerOptionDefault
CF Durable Object (DurableObjectAgentBase)maxStreamChunkBytes on the DO config1.8 MB (DEFAULT_MAX_STATE_BYTES)
DOStreamManager directlymaxChunkBytes1.8 MB
store-cloudflare DurableObjectStreamManager (and its StreamServer)maxChunkBytes1.8 MB (DEFAULT_MAX_STREAM_CHUNK_BYTES)
InMemoryStreamManagermaxChunkBytesInfinity
RedisStreamManagermaxChunkBytesInfinity
  • The Cloudflare limits must be at most 2 MB (CF_SQL_MAX_VALUE_BYTES / STREAM_DO_MAX_VALUE_BYTES). Anything else makes the constructor throw a typed validation_error.
  • The store-cloudflare binding sends its limit with every write and end, and the StreamServer enforces it. Deploy them together, as before.
  • Memory and Redis declare no limit, so a large tool result that streams today keeps streaming. Configure maxChunkBytes to reproduce a production limit in tests.

The split ​

Every runtime's model-attempt writer is wrapped with core withChunkSplit (runtime-js and the CF DO, CF Workflows, Temporal, DBOS). It cuts three chunk kinds into consecutive pieces:

  • a text_delta;
  • a streaming thinking delta (not the isComplete block);
  • a tool_arg_stream_delta.

Each piece is at most 64 KiB (or half the stream's limit, when that is smaller), measured as stored (JSON-escaped UTF-8) and cut on code-point boundaries. Pieces keep the chunk's run, attempt, step and agent. Readers, including stock useChat, append deltas, so the text is identical.

A chunk carrying remoteSource (proxied from a remote child) is never split: the child split it at its own source, and the parent stream's dedup keys on its child sequence (RM-86).

If you write your own model loop (bring-your-own loop), wrap your attempt writer the same way: withChunkSplit(writer, splitPieceBytesFor(streamManager.limits)).

Custom StreamManager implementations ​

The store contract (streamManagerContractTests) adds the CS.* cases and two options:

  • expectedDefaultMaxChunkBytes (required): your store's default limits.maxChunkBytes. Use Infinity for no limit.
  • createManagerWithLimit(maxChunkBytes) (optional): build your manager with a finite limit, so the refusal and omission cases (CS.end-oversized-parts-omitted, …) run.

To pass them:

  • Writes: measure the JSON you bind with core assertStreamChunkFits(streamId, json, limits) after your drop decisions and BEFORE allocating a sequence.
  • Ends: never refuse one. Fit it with fitStreamEnd(finalOutput, terminal, limits), store only the parts it kept, persist its omitted (report it as metadata.endOmitted, and on any end frame you write as omitted), and warn with warnStreamEndOmissions. parseStreamEndOmitted / streamEndOmittedFrom read a stored or wire value back.
  • The declared limit: expose it as limits (core streamLimits(maxChunkBytes, { platformMaxBytes }) validates it).

Released under the MIT License.