Example App Testing Strategy

This guide documents how the example app E2E tests (examples/opennext-cloudflare-do/tests/*.spec.ts, examples/research-assistant-cloudflare-do/tests/*.spec.ts) are designed and what discipline contributors must follow when modifying them.

Read this before adding, modifying, or "fixing" any example test.

The principle: mock as little as possible

The example tests exercise the SDK end-to-end against a near-production stack:

• Real wrangler dev with workerd (the production Cloudflare runtime, not a test double)
• Real Cloudflare Durable Object emulation
• Real D1 SQLite storage
• Real Workers AI / Vectorize bindings (when used)
• Real next.js production build served by OpenNext
• Real Playwright browser (Chromium) driving the UI
• Real Server-Sent Event streams over HTTP
• Real network refresh / reload semantics (page.reload())

The only thing mocked is the OpenAI HTTP API, and only because real LLM responses are nondeterministic in ways that have nothing to do with what the SDK is doing. Everything else runs as it would in production.

This is intentional. Examples are the place where bugs that only manifest under the full production runtime get caught. A bug in stream resumption that only shows up when workerd serves SSE through a specific code path will not be caught by mocking workerd.

What "highly realistic mocks" means here

The OpenAI mock is not a hand-crafted set of "happy path" responses. It is a record/replay HTTP proxy:

1. Record mode forwards each new request verbatim to the real API (api.openai.com, or a LiteLLM / OpenAI-compatible proxy in front of it) and persists the actual response body to a fixture file. A request whose fixture already exists is served from it, not forwarded: specs that open with the same turn then see, while recording, the reply that replay will serve them. Re-recording that turn would change the reply under a follow-up turn recorded after the first one, and the follow-up would never match on replay. The same holds for parallel workers: a request that arrives while its hash is still recording waits for that recording and gets the same reply, without a second upstream call. A recorded hash is served exactly as replay serves it, paced and with its stream gate. So a spec that needs a gate to reach its next request (e.g. a reply held mid-text) records one more turn on each repeated recording run, until it passes.
2. Replay mode serves the recorded fixture for the same logical request.

Fixtures are real OpenAI Responses API SSE streams, captured from a real model responding to a real prompt with real tool definitions. They include all the quirks, partial JSON streaming, and failure modes that the real API exhibits.

The proxy is one shared module, examples/openai-vcr/ (a private workspace package with its own unit tests); each example's test-utils/openai-mock-server.ts is a three-line CLI over it that serves that example's test-utils/openai-fixtures/.

This matters because the SDK's job is to handle whatever a real LLM returns. If we hand-crafted "clean" fixtures, the tests would only prove that the SDK works against fictional LLM behavior. Real LLMs sometimes emit:

• Tool calls with empty argument JSON before the full payload streams
• Multiple parallel tool calls in a single response
• Reasoning tokens before the actual response
• Error mid-stream that the SDK has to recover from
• Different status codes for retries

All of those need to be in the fixtures because all of those happen in production. Hand-crafted fixtures would lie to us.

Hard rules — no shortcuts allowed

These are not suggestions. Code review will reject changes that violate them.

1. Fixtures MUST come from real OpenAI

Every JSON file in test-utils/openai-fixtures/ must have been produced by OPENAI_MOCK_MODE=record against the real model API, directly or through a proxy that forwards to it. Do not write fixture JSON by hand. Do not modify fixture bodies. Do not synthesize SSE streams.

Fixtures are published with the repository, so they carry nothing about the account that recorded them. A fixture keeps the request's path, method and body, the response status and body, and only the content-type response header. The recorder refuses (a 500, nothing written) a response that contains its API key or a proxy's hostname, and examples/openai-vcr's fixture-audit test fails on any committed fixture with another header or a key-shaped string.

If you change a test prompt or modify the agent's system prompt / tool definitions, the existing fixture is stale — re-record it. See "Re-recording" below.

2. The mock MUST stay structural, not selective

The proxy hashes requests by their semantic content (model + messages + tools), with non-deterministic fields (call_id, timestamps, epoch ms, "Today's date is X") normalized away. It does not match by prompt substring, "first user message contains X, return Y." That kind of selective matching encodes "what we wish the LLM would do" rather than "what it actually does," which is the shortcut this whole architecture exists to prevent.

If you find yourself wanting to add a "if request matches pattern, return canned response" branch to the mock, stop. That is the line between a record/replay proxy and a stub. We're not crossing it.

Timing controls are not selective matching: they never choose or change a response, only when it is written. The mock has three, all documented in examples/opennext-cloudflare-do/test-utils/README.md: stream gates ([mock-gate:<id>], a held text response; [mock-gate:<id>@<n>] holds after n text deltas instead of the default, for a reply whose length the test does not control), the request log (GET /__log, which model requests arrived) and armed holds (POST /__hold, the first request whose body contains a given text is held before its first byte until the test releases it). A test asserts a hold's or gate's state rather than inferring it.

3. Tests MUST assert SDK invariants, not LLM behavior

The SDK's invariants are:

• Snapshot consistency: re-fetching the same session returns the same data
• Resume idempotence: page refresh during/after streaming does not duplicate or lose messages
• Stream completion: every started stream eventually closes with status: ended
• Tool result threading: tool calls and results pair correctly across refresh boundaries

These are deterministic for a given LLM response stream. They hold regardless of whether the LLM made 1 tool call or 12.

What the tests must not assert:

• "LLM made exactly N tool calls" — that's testing the model
• "LLM said the word 'time' in its response" — that's testing the model
• "LLM returned a JSON-shaped argument" — that's testing the model

If a test prompt was "Call get_current_time exactly once" and the recorded fixture shows the model called it 6 times anyway, the correct test assertion is "consistency of the count across refreshes," not "the count is 1." The fixture captures what the LLM actually did; the test verifies the SDK handled it correctly.

4. Test prompts MUST be natural language, not strict directives

Old-style strict prompts:

// ❌ Don't write tests like this. Real LLMs ignore these directives ~13%
// of the time, and the test reduces to "is the model's mood good today."
'Call get_current_time EXACTLY ONCE to tell me the time. EXACTLY 1 tool call, no more.';

Natural prompts:

// ✅ The LLM responds predictably-ish, and the recorded fixture locks
// in whatever it actually did. Subsequent runs replay that recording
// deterministically.
'What time is it? Just tell me once.';

The "Just tell me once" / "briefly" / "in one sentence" hints are fine — they steer the model toward the kind of response real users would write. Strict imperatives ("EXACTLY", "MUST", "DO NOT") are not, because they're testing the model's compliance behavior rather than the SDK.

5. NEVER add test.skip to dodge a real failure

test.skip is only acceptable when a test exercises a path that the mock infrastructure provably cannot make deterministic, and the skip is paired with:

• A clear comment explaining the specific reason (cite the symptom, not "flaky")
• A TODO that names what would need to change to re-enable it (e.g., "needs hand-crafted fixtures + timing-invariant rewrite")
• A linked issue or note in test-utils/README.md

If the test is failing because the assertion is wrong, fix the assertion. If it's failing because the SDK has a real bug, fix the SDK. Do not skip to make CI green.

Quarantined tests

This is the single register of every quarantined test in the repo: example-app Playwright specs and packages/e2e tests. A quarantine is the one exception to this rule: a user decision to test.fixme a test (or it.skip it in vitest, which has no fixme, or set the harness's skip gate) because it fails on a known SDK bug. The test stays in the file, the bug keeps an owner, and the un-quarantine is part of the fix. Each quarantined test has a comment that names the bug, the follow-up entry and the owner. The follow-up entry in follow-ups.md lists the tests and the exit condition. Add every new quarantine to this table.

Tests	Bug	Follow-up	Owner	Exit condition

6. The mock MUST cover ALL OpenAI calls a test makes

If a single test triggers 30 OpenAI requests and you only have 29 fixtures, the missing one will hit the mock's miss path and dump a diagnostic file (_miss-<hash>-<timestamp>.json). That file is ignored by git on purpose — it's a signal that recording was incomplete, not something to commit and hope nobody runs that path.

Re-record until the test runs end-to-end with zero misses in replay mode.

Test architecture diagram

┌─────────────────────────────────────────────────────────┐
│                 Playwright Browser                       │
│           (real Chromium, real refresh)                  │
└──────────────────────┬──────────────────────────────────┘
                       │ HTTP
                       ▼
┌─────────────────────────────────────────────────────────┐
│              wrangler dev → workerd                      │
│   (real Cloudflare Worker runtime, real DOs, real D1)   │
│                                                          │
│  OPENAI_BASE_URL=http://localhost:3030/v1 and the       │
│  dummy key test-mock-key (--var or .dev.vars), always   │
└──────────────────────┬──────────────────────────────────┘
                       │ HTTP (POST /v1/responses)
                       ▼
┌─────────────────────────────────────────────────────────┐
│       openai-mock-server.ts (port 3030)                  │
│       → examples/openai-vcr                              │
│                                                          │
│  RECORD mode:  forwards to ITS OWN OPENAI_BASE_URL       │
│                  with ITS OWN OPENAI_API_KEY,            │
│                  saves response to fixture               │
│  REPLAY mode:  matches by semantic hash, serves         │
│                  fixture verbatim                        │
└──────────────────────┬──────────────────────────────────┘
                       │ (record mode only)
                       ▼
     api.openai.com, or a LiteLLM / OpenAI-compatible proxy
              (the actual LLM)

Only the mock ever holds a real key or the upstream URL. The app under test always gets the mock's URL and the dummy key, in replay and record alike, so an OPENAI_BASE_URL / OPENAI_API_KEY exported in your shell (for the recorder) never reaches the app. Every harness pins this (examples/openai-vcr's harness-pinning test checks the Playwright configs, remote-agents-cloudflare-do/test-utils/start-workers.sh and the CI jobs). And a replay run never reuses a mock left running in record mode: each config's health probe is /healthz?mode=replay, which a record-mode mock answers 409. (An app server YOU started by hand with real credentials is still reused; stop it before a replay run.)

Everything below "Playwright Browser" runs identically in CI and locally, including the full SSE streaming chain. The only thing that differs is whether the mock is in record (developer machine, populating fixtures) or replay (CI, deterministic).

Re-recording

When test prompts, agent system prompts, or tool definitions change, the recorded fixtures become stale (the semantic hash will miss). Re-record.

The recorder needs a real key, and optionally a proxy:

Variable	Read by	Meaning
OPENAI_API_KEY	the mock only	The key it sends upstream (it replaces whatever the app sent). Required in record / passthrough mode.
OPENAI_BASE_URL	the mock only	Upstream base URL including /v1. Default https://api.openai.com/v1; set it to use a proxy.

Give them to the mock process alone, from your secrets manager or a gitignored env file, never on the command line and never in .dev.vars (e.g. infisical run -- <mock command>, op run -- …, or set -a; . ~/.config/your-llm.env; set +a in a subshell). The mock never logs either value; it logs upstream=openai or upstream=custom. It refuses to start in record mode without a key, or with an upstream that is the mock itself.

cd examples/opennext-cloudflare-do

# 1. Wipe stale fixtures
rm -f test-utils/openai-fixtures/*.json

# 2. Mock in RECORD mode. OPENAI_API_KEY (and OPENAI_BASE_URL for a proxy) reach
#    the mock's environment only, from your secrets manager, e.g. Infisical:
OPENAI_MOCK_MODE=record infisical run --silent -- \
  node --experimental-strip-types --no-warnings test-utils/openai-mock-server.ts &

# 3. The worker, pointed at the mock with the dummy key (as in replay)
pnpm run build:worker
pnpm exec wrangler dev --local \
  --var OPENAI_BASE_URL:http://localhost:3030/v1 --var OPENAI_API_KEY:test-mock-key \
  --var DISABLE_MEMORY:true --var ENABLE_TEST_FAULTS:true &
until curl -s http://localhost:8787 >/dev/null; do sleep 1; done

# 4. Run the tests: fixtures populate as tests hit endpoints. Playwright reuses
#    the servers already listening; OPENAI_MOCK_MODE=record makes its health
#    probe accept the record-mode mock (a plain run refuses to reuse one)
OPENAI_MOCK_MODE=record pnpm run test:e2e

# 5. Verify the recording is clean. Inspect a fixture:
ls test-utils/openai-fixtures/ | head -5
cat test-utils/openai-fixtures/<hash>.json | jq '.request.body | fromjson | .input'

# 6. Run again in REPLAY mode (no key anywhere) to confirm zero misses
pkill -f "wrangler dev"; pkill -f openai-mock-server
pnpm run test:e2e   # playwright starts the replay mock + worker itself;
                    # must pass with zero `_miss-*.json` files left behind

# 7. Check the fixtures are publishable, then commit fixtures + test changes
pnpm --filter @helix-agents/example-openai-vcr run test
git add test-utils/openai-fixtures/ tests/
git commit -m "test(opennext): re-record fixtures for <reason>"

The other examples differ only in how the app starts (their playwright.config.ts and test-utils/README.md): approval-gates runs next dev, research-assistant-cloudflare-do runs wrangler dev, and remote-agents-cloudflare-do starts both of its workers with test-utils/start-workers.sh.

If step 6 leaves any _miss-*.json files, the recording is incomplete. Common causes:

• Test took a different code path the second time (real LLM may have returned a different response between runs, leading to different follow-up requests)
• Hash normalization missed a non-deterministic field — inspect the miss file and the closest-matching recorded fixture, find the field that differs, and either extend normalizeForHashing in examples/openai-vcr/src/hashing.ts (a hash change: re-record every example) or accept the test's nondeterminism as a test-design issue (rare; almost always there's a normalizable field at fault)

Do not "fix" misses by re-running record mode over the existing fixtures: record mode keeps every fixture it finds and only adds the missing hashes, which hides the underlying non-determinism. Wipe openai-fixtures/ and re-record cleanly.

Adding a new test

1. Write the test against a natural-language prompt
2. Make the test assert SDK invariants only (consistency, no duplication, snapshot stability)
3. Run in record mode locally — fixtures populate
4. Run in replay mode locally — confirm no misses, deterministic pass
5. Commit fixtures + test together

Do not commit a test without committing its fixtures. CI runs in replay mode; an unrecorded test will hit the mock's miss path and fail.

CI

CI runs the example tests in OPENAI_MOCK_MODE=replay against checked-in fixtures. The CI script in ci/test-matrix.yml spawns openai-mock-server.ts alongside wrangler dev before running playwright.

A miss in CI means a fixture is missing or stale, and the test fails with a clear mock_fixture_missing error pointing at the hash. The fix is always: pull the branch locally, re-record, push the fixtures. There is no production-equivalent fallback to real OpenAI in CI. The replay jobs hold no key at all: they give the app test-mock-key.

Two example suites (nextjs-redis, resumable-streams-nextjs) and the test:live-llm job (the live vitest files) DO call a real model. They share the .live_llm template in ci/test-matrix.yml: a manual button on same-repo MRs (never blocking, never started automatically), with OPENAI_API_KEY (required), OPENAI_BASE_URL, ANTHROPIC_API_KEY and ANTHROPIC_BASE_URL (optional) from masked CI/CD variables scoped to the e2e-tests environment. test:live-llm also runs a Temporal dev server as a service (TEMPORAL_ADDRESS=temporal:7233) for its live Temporal case.

Anti-patterns to call out in review

Anti-pattern	Why it's wrong	Correct alternative
expect(toolCount).toBe(1)	Tests LLM compliance, not SDK	Assert consistency: expect(snap2).toEqual(snap1)
'Call get_current_time EXACTLY ONCE'	Strict prompt that LLMs ignore	Natural prompt: 'What time is it?'
Hand-written fixture JSON	Doesn't match real LLM behavior	Re-record from real OpenAI
// match: prompt contains "time" → return canned response in mock	Selective matching = hidden stub	Use semantic hash + recorded fixture
test.skip without comment + TODO	Hides real failures	Either fix it or document specifically why it's deferred
expect(text).toContain('hello') (text from LLM)	Tests LLM output	Test SDK behavior; if you need text presence, assert the SDK didn't drop it via length > 0
Bumping per-test setTimeout to 5+ minutes	Hides slow paths	Profile the slow path; if it's legitimate, document it; if it's not, fix it
Adding retries to mask flakiness	Hides nondeterminism	Find and fix the source of nondeterminism
Typing (fill) right after page.goto on a controlled input	Races React hydration	Wait for the page's app-ready marker first (see below)

Wait for hydration before typing

A controlled React input ignores text typed before React hydrates the page: the DOM shows the text, the component's state stays empty, and a button gated on it stays disabled. Under next dev on a loaded CI runner hydration can land seconds after page.goto returns, so a test that types at once flakes. That was FU-FLAKE-APPROVAL-GATES-PLAYWRIGHT: 18 of 50 approval-gates jobs, hidden by retries: 1.

An example page that tests type into exposes an app-ready marker, set in a useEffect (it only runs after hydration), and every test waits for it before typing:

const [hydrated, setHydrated] = useState(false);
useEffect(() => setHydrated(true), []);
// <form data-testid="chat-form" data-hydrated={hydrated ? 'true' : 'false'}>

await page.goto(BASE_URL);
await expect(page.getByTestId('chat-form')).toHaveAttribute('data-hydrated', 'true');
await page.getByTestId('message-input').fill('…');

approval-gates/tests/hydration-guard.spec.ts proves the wait is what makes typing land. It holds the JS chunks back and returns from page.goto at domcontentloaded. Without the wait, Send stays disabled every time.

wrangler dev's dropped POST: the one sanctioned re-send

Under runner load wrangler dev stalls, and its ProxyWorker loses the connection to the UserWorker. It retries the GETs in flight but answers a POST with its own plain-text 500 (Error: Network connection lost., logged as "Error inside ProxyWorker … failed after 1 attempt"). That request never reached the chat route, whose failures are typed JSON, and production has no ProxyWorker (FU-FLAKE-OPENNEXT-PROXYWORKER-DROPPED-POST).

The opennext specs re-send such a send through examples/opennext-cloudflare-do/test-utils/dev-proxy-dropped-send.ts. The re-send happens once, is logged, and goes under the send's own X-Helix-Request-Key. It is allowed only when the answer is exactly that 500 AND the snapshot shows the turn NOT committed. Any other failure, a committed turn, an unreadable snapshot or an unkeyed send fails the spec. It is not a retry policy: never widen it to other answers. The decision is unit-tested in tests/dev-proxy-dropped-send.spec.ts.

Current coverage status

CC57 — 100% of example E2E tests now follow the discipline. Every spec file in both examples/opennext-cloudflare-do/tests/ and examples/research-assistant-cloudflare-do/tests/ has been converted to natural prompts + SDK-invariant assertions, and both examples route OpenAI calls through the VCR-style mock at test-utils/openai-mock-server.ts. CI runs the full suite in OPENAI_MOCK_MODE=replay against committed fixtures and is independent of OPENAI_API_KEY for all checked-in tests. No testMatch restrictions remain.

opennext-cloudflare-do — 12 spec files, all on the VCR mock (the first 7 converted in CC57):

• api-consistency.spec.ts — 5 tests (all enabled; 2 previously skipped mid-stream-refresh tests were rewritten to be timing- invariant and re-enabled in CC57). The RM-08 quarantine of 1 was removed by S5's snapshot MR.
• brief-flow.spec.ts — 7 tests, marker-based assertions replaced with content-preservation invariants (text length doesn't shrink across refresh; tool counts stable; both deterministic agent tools surface).
• comprehensive-refresh.spec.ts — 9 tests, "CRITICAL INSTRUCTIONS" prompts and expect(text).toContain('1, 2, 3, 4, 5') numeric- sequence checks replaced with snapshot-stability + part-order invariants. The RM-08 quarantine of 3 was removed by S5's snapshot MR (15/15 at --repeat-each=15 --retries=0).
• follow-up-message.spec.ts — 1 test, "EXACTLY ONCE" prompts replaced with natural prompts; the load-bearing assertion (no tool-input-delta console errors) preserved.
• mid-text-reload-scoring.spec.ts — 23 tests (RM-08; design spec §3.4 client.*). 20 iterations reload while the assistant streams text and require the page to render exactly the final snapshot (client.no-loss / client.no-duplication, classified LOSS / DUPLICATION on failure) and the reconnect GET to carry exactly the rendered snapshot's cursor and resumeMessageId (client.cursor-and-id-match). Plus a reload while paused on the colorPicker client tool (client.resumes-on-paused), a server-side snapshot read that fails 3 times and still resumes (client.retries-5xx-snapshot), and every read failing → the explicit error state with Retry, never an empty chat, whose browser-side Retry is itself retried (client.error-after-exhaustion). No new fixtures: it reuses the recorded first-turn-refresh-mid-stream and client-tool-multiturn chains. The mid-text window is deterministic: each iteration's prompt carries a mock stream gate ([mock-gate:<id>], see test-utils/README.md), the mock holds the text step after its first 40 text deltas, and the test releases it only after the reloaded page has rendered the live snapshot and re-attached. (Replay pacing alone lost that race on slow CI runners: the reload's document request was served after the run had ended.) It needs the worker's ENABLE_TEST_FAULTS=true (the test-only x-helix-test-snapshot-faults header); playwright.config.ts and CI set it.
• incident-replay.spec.ts — 3 tests, the 2026-10-17 writer-chat incident (a dropped send that never re-anchored, CP-13) on a stock useChat page (/incident/[sessionId]). The first: a send stream reset by the drop proxy (which matches SSE frames in the decoded chunked body), the re-attach (useReanchorOnError, the model's reply held mid-text by [mock-gate:<id>@10]), client tools emitted during a network outage, their deadline firing with no request (observed in the mock's request log while nothing reaches workerd), a reload that re-attaches to the still-running run (on the CF DO the parent awaits its remote fact-checker inside the step), and a send over that running run refused 409 state_already_running, kept, and sent once after the run ends. The second (CP-167): a send cut before its first frame still re-attaches. The third (/incident-companion/[sessionId]): the writer consults a non-blocking persistent fact checker and asks the user in one step; a send while the parent is paused on askUser and the companion runs is taken once (the pending call abandoned), and the companion's verdict reaches the parent's next turn once (read from the companion's own session; the parent DO's refs have no public read). Each leg is a test.step with soft assertions. The lost-remote-child leg is FU-INCIDENT-E2E-REMOTE-CHILD-LOST.
• live-stream-client-error.spec.ts — 1 test (DI-20). Asserts in-page, on one clock, that the chat-error banner never appears and the assistant text renders from the live stream before any /snapshot refetch. End-state assertions are blind to a live client stream error, because a later refetch/reload repaints the finished messages. Verified against a mutant build that added one extra key to text-delta: this spec failed by name ("Type validation failed"), 12 other specs failed only incidentally with generic messages, and 22 passed blind. It reuses the recorded "What time is it? Just tell me once." fixture.
• interleaved-content.spec.ts — 4 tests, "CRITICAL INSTRUCTIONS" prompt and >=2 tools + 2-of-3-phrase-presence assertions replaced with conditional invariants ("if the LLM produced N tools, all are completed; text length doesn't shrink across refresh"). The RM-08 quarantine of 3 was removed by S5's snapshot MR.
• reasoning-display.spec.ts — 2 tests, was already mostly natural; strengthened the post-refresh assertion to require text length is non-decreasing.
• stream-resume-consistency.spec.ts — 4 tests, "EXACTLY ONCE" / "EXACTLY 2" prompts and .toBe(1) tool-count assertions replaced with consistency invariants (toolCountsAfter ≤ toolCountsBefore + 1 mid-stream; equal post-completion).

research-assistant-cloudflare-do — 2 spec files, both converted:

• research-assistant.spec.ts — 10 tests + 1 previously skipped stream test re-enabled (mock makes it deterministic). Routes through the mock now; CI no longer depends on OPENAI_API_KEY.
• sub-agent-delegation.spec.ts — 8 tests, INSTRUCTIONS: / You must prompts replaced with natural prompts ("Use the summarizer to give me one short summary"). LLM-compliance asserts (subagentStarts.length > 0) replaced with conditional SDK invariants ("IF the LLM fired the sub-agent, THEN the SDK pairing / isolation / propagation contracts hold"). Pairing invariant (starts.length === ends.length) and terminal-event invariant (subagent_end < end) hold unconditionally.

Re-recording the full set

When prompts, agent definitions, or tool definitions change in either example, re-record fixtures using the per-example procedure documented above. Both examples now have their own test-utils/openai-fixtures/ directory; they are independent — re-recording one does not affect the other.

To convert another spec file:

1. Read this whole document
2. Open the spec file and audit every expect(...).toBe(N) involving tool counts, message counts, or LLM text. Convert to invariant assertions (toBeGreaterThanOrEqual, toEqual(snap1), etc.)
3. Audit every prompt for "EXACTLY", "MUST", "DO NOT", strict compliance directives. Convert to natural language with mild hints ("just tell me once").
4. Re-record fixtures for the new prompts (procedure above)
5. Verify replay passes with zero misses
6. Update playwright.config.ts's testMatch to include the file
7. Update this doc's "Current coverage status" section
8. Commit fixtures + spec changes + config change in one commit

Why this matters

These tests are the closest thing we have to "does the SDK work in production?" The whole point is that they catch bugs that only show up in the full production stack. Every shortcut taken (a hand-crafted fixture, a stubbed code path, an "EXACTLY" prompt that masks a real bug under "the LLM was non-compliant") reduces the value of the tests until they're verifying nothing.

The mock infrastructure exists specifically so we can have deterministic, fast tests that still exercise every byte of real production code except the LLM HTTP call. Don't spend that deterministic budget on shortcuts.