One document, two models, and arithmetic in between

Read the two middle columns against each other. The document is written on almost every row; speech happens on very few. That gap is the architecture — and the version before this one had no such gap, because deciding what to say was the job of every frame.

The document is one JSON file, so load → mutate → save must be atomic and there is exactly one lock. Originally it was held across the whole tick, which meant a tick owned the document for two model round trips plus any web search they made. Measured at the point where a question arrives mid-reasoning, that cost the user two full seconds of nothing.

Two consequences follow, and both are load-bearing. Every window reloads from disk on entry — a window that reuses a document read before a model call silently rolls back whoever wrote in the meantime, a lost update that leaves no trace because both writes reported success. And the document's revision number is bumped on window entry, not on save: windows are serialized by the lock, so entry order is write order, and the browser can drop any render older than one it has already painted.

There is no retrieval step and no memory tool. Every prompt is built by rendering the document, because a model that has to ask for its state will reliably forget to ask. The section order below is not cosmetic.

The order is stability-first. Title, tasks, narrative and environment facts change rarely; raw captions change on every tick. So the things that move least go at the top, and consecutive ticks share the longest possible unchanged prefix — which is what the reasoning provider's prefix cache is able to charge less for. Ordering the document by what a reader would find natural would have cost real money on every frame.

The caption buffer is bounded at 24. On overflow the oldest sixteen are summarised into the narrative by one cheap model call, with time spans preserved rather than only facts, plus any durable environment facts worth promoting. Raw captions are never silently dropped, and the freshest window is never compacted.

Wall-clock arithmetic is free; tokens are not. The trigger check runs on every tick and every 20-second poll, costs nothing at all, and the reasoning model is only woken when it returns something — or when a frame arrived, or the user spoke.

Each one claims its own state on firing, before anything downstream awaits. An unclaimed trigger fires again on the next tick while the first one is still thinking — the same double-fire lesson the timer system taught in v1.

Unprompted speech is then gated by a 90-second politeness budget, and exactly two things bypass it. [URGENT] — physical risk, or work about to be ruined — and the follow-up window that a user's own question opens.

“Find the chicken and the onions” opens a search. Every open item is put to the camera by name, in one block, on every frame — one block rather than one question per item, so the list never competes with the four-watch cap and a fourth search cannot silently stop reaching the camera.

chicken packet: FOUND — middle shelf, behind the milk
onions: NOT VISIBLE — this part of the frame is the door rack

There is no tool that marks an item found. The model can open a search and cancel one; only plain string matching over the caption — zero tokens — sets the found state. That is what stops an inference standing in for an observation. The caption said “several bags of lentils” and the model once upgraded it to “I can see the beans”. The found state now has no model-facing door.

Two smaller decisions here both came from thinking about what happens when something goes wrong. Found and announced are separate flags— one is about the world, the other about speech — so a failed utterance simply re-fires next tick instead of leaving an item found and unspoken forever. And matching is by name, never by question index: the index is positional over a list rebuilt each tick while the vision call runs with the lock released, so eventually it would announce one item's location under another item's name.

In this split the vision call is the only image cost, so its prompt tokens are the per-frame bill — which is what makes the number knowable at all. A tick is roughly 1,350 input and 90 output tokens at the coarse tier; the close-up tier caps the longest side at 1,024 px, which is about a thousand image tokens on its own.

v1 survives on a free tier because its cadence is slower and most of its turns are text-only. v2 ticks continuously by design — it is a fundamentally heavier vision consumer, and the honest conclusion was that the free tier is not a constraint it can be engineered inside. That is now enforced rather than documented: the wrong provider refuses to start.

The close-up tier gets a nudge, then a backstop. The count of close-up frames is rendered into the document, so the drift is visible to the thing causing it, and a hard cap forces coarse at 120. v1 proved that a fine mode set once is never voluntarily reverted; v2's own live session proved the opposite failure, and the tier was never chosen at all.

The document-mutating tools close over the agent's current in-memory document, so a turn's tool calls and the agent's own writes can never interleave on disk. And retract_environment_fact takes a correction, not just a deletion — the raw captions that produced the wrong inference are still in the buffer and will suggest it again on the very next tick. A hole in the fact list does not block that; a durable “the bag on the pantry shelf is NOT toor dal” does.

The duplicate check is the one I would highlight. It was a reasonable idea, it was built, and measuring it is what killed it — in both directions at once. Removing it also removed the pressure it existed for, because form and safety moved into a tool that replaces rather than appends, which makes duplicates impossible by construction instead of by threshold.