Repository navigation
Observer prompts miss the prompt cache because the first line is the current minute - #103
Merged
Seopril merged 2 commits intoNov 2, 2026
Merged
Conversation
Collaborator
|
Thanks for taking the time to open this PR. I reviewed it and it looks good to me. Would you mind rebuilding the dist/index.ts? I'll get it merged once that's done. I'll also be reviewing our pipeline to avoid having to do this for commits/pipelines that aren't being directly published to npm, so apologies for that. |
The observer user message opened with the minute-resolution local time, so its prefix changed on every run and the reflections and observations after it missed the provider prompt cache. Move the time line below them, ahead of the instructions, so the instruction's 'current local time above' reference still holds.
rnavarro
force-pushed
the
pr/observer-timestamp-cache-prefix
branch
from
November 1, 2026 23:55
52b007e to
998fc63
Compare
Contributor
Author
|
Thanks for the feedback, updated! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I was checking prompt cache hit rates across my pi sessions and found the memory workers well behind the main agent loop: about 72% of input tokens read from cache for worker calls, against 95% for the main loop on the same model. Observer requests were the worst. A typical observer prompt of tens of thousands of tokens read only the shared system prompt from cache, even when its reflections and observations were the same as the previous run.
Diffing consecutive observer requests, the first byte that differed was nearly always the time on the first line of the user message, for example
19:04vs19:06. The observer user message insrc/agents/observer/agent.tsopens withCurrent local time: ${now}, andnowTimestamp()has minute resolution, so the prefix changes on almost every run and the reflections and observations after it are sent as uncached input.This PR moves the
Current local time:line below the reflections and observations, just ahead of the instruction paragraph, so those blocks stay in a stable prefix between runs. The instruction text says "fall back to the current local time above", which is still accurate, so no prompt wording changes.A test in
tests/observer.test.tschecks that the user message starts withCURRENT REFLECTIONS:and that the time line sits after the prior observations and beforeNEW CONVERSATION CHUNK:. It fails on the current layout. I also ran it in a live session and confirmed the observer request now starts withCURRENT REFLECTIONS:.The
dist/index.tsdiff is larger than this change: CI on master has failed itsgit diff --exit-code -- diststep since #93, and a freshnpm run buildalso picks up thesrc/changes the committed bundle was missing.#96 also edits this user message and keeps the timestamp first. If #96 lands first, I can rebase this onto it.