Skip to content

An aborted model call reports its input usage (#180) ​

A model call cut short by a Stop or interrupt, or failed after the provider started answering, now carries the usage the provider reported before the cut on its result. Before this release it carried none, so the input tokens the provider billed for it were never booked. This is a minor release of @helix-agents/llm-vercel and a patch release of @helix-agents/core (documentation only). Nothing is removed or renamed.

What changes ​

  • The result of such a call (the partial text of a call stopped mid-stream, or an error result), the afterLLMCall hook's usage and the usage store's tokens entry now carry the call's input side:
    • promptTokens, the total input: uncached + read from cache + written to cache, the same value a completed call reports;
    • cachedTokens and cacheWriteTokens, when the provider reported them.
  • It is a lower bound. The provider reports the output count only at the end of the stream, so the output generated before the cut is not known. completionTokens and totalTokens are absent (not 0). A provisional output count sent at the start is not used.
  • No step_end usage is written for the cut call.

Which calls ​

ModelA call cut short reports
@helix-agents/llm-vercel with an @ai-sdk/anthropic model objectits input and cache tokens
The same, through an Anthropic-format proxy (for example LiteLLM's /v1/messages)its input and cache tokens
An OpenAI-compatible route (/chat/completions): usage arrives only in the last chunknothing
A gateway model id string, or a V2 model objectnothing

What to check ​

  • Cost and budgets: compute them from the per-kind fields (prompt, completion, cached, cacheWrite), never from total. After an interrupted call, a session's tokens.total sums only the calls that reported a total, so it is less than prompt + completion.
  • Code that read a cut call's missing usage as "nothing was billed": its result now carries usage. Book it. A stopped call's partial text result carries input-only usage, with no marker that it is partial; read completionTokens being absent as "output not reported".
  • A custom adapter wrapper: keep forwarding the result's usage unchanged.

See the usage guide, Calls cut short.

Released under the MIT License.