Repository navigation
fix(gmail): read every history page before advancing the watch cursor - #1312
saariuslystoned wants to merge 2 commits into
Conversation
watch serve and watch pull made one users.history.list call (100 records) per notification and then stored the response historyId, which is the mailbox's current history ID. Records after the first page were never fetched, and the stored cursor skipped them for good. Gmail's reference says the historyId can be stored once no nextPageToken is returned. Page through the history with the repeated-token guard before the processor advances the cursor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
🦞👀 Pull request received. I will update this pull request when review starts. ClawSweeper review completeClawSweeper finished reviewing this revision. The review result is being finalized. |
|
Codex review: needs maintainer review before merge. What this changesFetch every Gmail history page before returning the cursor used by watch push and pull processing. Example: Cursor 100 yields m1 on page one, m2 on page p2, and mailbox history ID 300.
Review scores
ProductKind: Bug fix · Worth it: Yes · Fix scope: Complete Merge readiness✅ Ready for maintainer review This PR fixes a message-loss bug still present on main; no actionable patch defects remain, and the previous changelog finding is resolved. Priority: P2 Before mergeNone. FindingsNone. Agent review detailsHow this fits togetherPush and pull consumers share a Gmail source adapter that converts history responses into records for message fetching, hook delivery, and cursor persistence. flowchart TD
A[Push or pull notification] --> B[Watch processor]
B --> C[Gmail source adapter]
C --> D[Gmail history pages]
D --> C
C --> B
B --> E[Message fetch and hook delivery]
B --> F[Saved watch cursor]
Technical reviewBest possible solution: Land the focused pagination repair in the shared Gmail source adapter. Do we have a high-confidence way to reproduce the issue? Current-main source shows the single-page request followed by cursor advancement; the contributor also records a main-versus-branch CLI demonstration, which this read-only review did not rerun. Is this the best way to solve the issue? The patch fixes retrieval in the existing shared owner, reuses the established pagination guard, and preserves processor behavior and filtering. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning medium; reviewed against e8428c36dd31. Provenance checked
TestingProof path: shipped entry point. SecurityNone. EvidenceWhat I checked:
Likely related people:
Preflight sourceSource unknown. Source comparison at the observed PR snapshot only. Completion is self-reported; body, policy, tests, execution, and report bytes are not verified. LabelsLabel changes:
Label justifications:
Rating scale6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior. WorkflowClawSweeper edits this one comment on every review. Comment HistoryReview history (1 earlier review cycle)
Reviewed November 8, 2026, 2:19 PM ET / 19:19 UTC (Revision 2). |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@clawsweeper re-review |
|
🦞👀 Re-review progress:
|
|
Thank you for the fix. I reproduced the claim and consolidated consuming every Gmail history page before advancing the watch cursor into #1336, with regression coverage and your coauthor credit retained. The full |
What problem this solves
For each Pub/Sub notification,
gog gmail watch serveandgog gmail watch pullcallusers.history.listonce, with the default 100 records (defaultHistoryMaxResults).gmailWatchSource.ListHistorynever readsnextPageToken. The processor then stores the response'shistoryIdas the new cursor. Theusers.history.listreference describes thathistoryIdas "the ID of the mailbox's current history record" and says:When a notification covers more than one page of changes, records 101 and later are never fetched. The stored cursor then moves past them, so the next notification starts after them too, and those messages never reach the hook. A burst of new mail, a bulk label change, or a
watch pullcatch-up after the consumer was down can each produce more than 100 records.Change
ListHistorypages throughusers.history.listwithpageTokenuntil nonextPageTokenremains. It uses the existingcollectAllPageshelper, which rejects a repeated page token and stops at its page limit with an error, so the processor never advances the cursor over unread history. The returnedhistoryIdcomes from the last page. The processor, theHistoryPagetype and--history-typeshandling are unchanged.docs/watch.mdnotes the behavior.Evidence
TestGmailWatchSourceListHistoryReadsEveryPageuses a two-pagehttptesthistory and expects two calls (the second withpageToken=p2), both records, andhistoryId300. Onmain(e8428c3) it fails with one call.TestGmailWatchSourceListHistoryRejectsRepeatedPageTokenexpects an error, not a cursor advance, when the token repeats. It fails onmain.TestGmailWatchServer_DeliversHistoryBeyondFirstPageposts a push notification to the watch server against the same two-page history and checks the hook payload. Onmainthe hook gets onlym1, and the stored cursor is still advanced to 300. With this change the hook getsm1,m2. The existing watch, history and processor tests pass.Built
gogbinaries frommainand from this branch were run against a local stand-in throughGOG_GOOGLE_API_BASE_URL, with an emptyGOG_HOMEand the file keyring. The stand-in is not Google: it servesm1on history page 1 withnextPageToken: p2,m2on page 2, andhistoryId: 300, plus a local hook receiver. Each run doesgmail watch start(cursor 100), startsgmail watch serve --token tok --hook-url <local> --fetch-delay 0, POSTs one push for historyId 300, then readsgmail watch status:On
main,m2is never fetched, and the saved cursor is already past it, so a later push will not pick it up.make cipassed locally (macOS arm64, go1.27.3).Boundary: no live Gmail mailbox or Pub/Sub subscription was used. The paging behavior follows the
users.history.listreference. Each notification now fetches every changed message in its range, which is the work the earlier behavior skipped.🤖 Generated with Claude Code