Repository navigation
fix: sync only changed rows on a learnings-fingerprint miss (GIT-98) - #38
Merged
2 commits merged intoSep 22, 2026
Merged
2 commits merged into
2 commits merged into
Conversation
…isses (GIT-98) On a Pro store the fingerprint misses on nearly every session_start: refresh_scar_behavioral_scores() rewrites each scar with >=3 recent usage rows and bumps its updated_at. Each miss re-downloaded the whole index, every row with its 1536-dim embedding. - A miss with a cache from this store downloads the manifest (id, updated_at; no vectors) with the bulk load's filters, order and limit. - Rows whose id and updated_at match are reused from disk; changed and new ids are fetched in full, 100 ids per request; rows gone from the manifest are dropped. The result is in manifest order, what a full download returns. - Any failure, an empty manifest, or no cache from this store: the full download, as before. cache-flush still always takes the full download. - The bulk load, fingerprint, manifest and delta fetch share one filter set. Venue (v1.8.0 setup.sql, 250 learnings, --usage 5): the start after a session_start drops from 921,780 B to 42,823 B — 245 reused, 5 fetched. +8 unit tests. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…rt (GIT-98) Exits 1 on 1.10.0 (the start after each session_start re-downloads ~922 KB); exits 0 with per-row delta sync. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A learnings-fingerprint miss no longer means a full download (GIT-98, per-row delta).
id,updated_atonly, no vectors, using the bulk load's filters, order and limit.updated_atmatch is reused from disk. Changed and new ids are fetched in full, 100 per request. Rows no longer in the manifest are dropped. The result is in manifest order, so it's exactly what a full download returns.cache-flushalways takes the full download.INDEXED_LEARNING_FILTERS), so they can't drift apart.Why
On a Pro store the fingerprint misses on nearly every
session_start.refresh_scar_behavioral_scores()rewrites each scar with 3 or more recent usage rows and bumps itsupdated_at. So each miss re-downloaded the whole index, every row with its 1536-dim embedding, to pick up a handful of changed scores.How to test
Standing rule: the venue runs the v1.8.0
setup.sqlfirst. The venuevujndhcumgroztfkkduswas provisioned fromv1.8.0:schema/setup.sql. Production was not touched. No schema change.250 learnings;
--usage 5gives 5 scars ≥3 usage rows, so eachsession_startrewrites 5 rows (confirmed on both rounds). Bytes received from the venue:origin/main1.10.0session_start245 reused from disk, 5 fetched, 0 droppedsession_start245 reused, 5 fetched−95% (921,780 → 42,823 B) for the start after a
session_start. Of the 42.8 KB, the manifest is 24,248 B (about 97 B a row) and the 5 full rows are 18,475 B. The manifest is now the floor, and it scales with the row count.New egress gate: the driver now fails if any start after the cold one logs a full index download.
mainexits 1 (two ~922 KB re-downloads); this branch exits 0. When the delta lands varies from run to run, because the in-process reload timing moves, but no start after the cold one pays full price.flowon this branch exits 0: close persisted, relevance readable, thread counts match.Unit tests: 1282/1282 pass (+8). The pure planning covers reuse, fetch, drop, rows with no
updated_at, and ids that vanish before the fetch. Through the realloadScarsFromSupabase: 5 changed rows fetch exactly those 5 ids and the result equals a full download, order included; inserts and deactivations; manifest failure falls back to the full download;cache-flushbypasses the delta; a cold cache does no manifest.Existing tests
None changed.
vector-disk-cache.test.ts"misses when the row count / newest updated_at changed" still holds:readVectorCachestill misses, and the delta is a separate path layered on top.Assumption (same as the fingerprint's)
updated_atmoves on every edit (setup.sql's trigger). An in-place edit that leavesupdated_atunchanged would be reused as-is. The fingerprint already couldn't see such an edit;cache-flushis the manual way out.Checklist
npm run test:unitpasses (1282)npm run test:e2e: not run; the venue driver is the e2e check hereBreaking changes
None.
🤖 Generated with Claude Code