Repository navigation
Conversation
#541) A drawing was read in full (scene JSON included) on every Excalidraw autosave by both indexers, and embedded as a title-only vector: the back-of-note extraction kept the frontmatter and Excalidraw's "switch to view" banner, so a blank drawing never looked empty. New files were also embedded immediately on create, with no debounce. - Strip frontmatter and the banner from the back of the note; bare `.excalidraw` scenes are non-embeddable (still findable by name). - Notes with no extracted text are skipped instead of embedded title-only, and the verdict is remembered per mtime (`emptyNotes`, like `failedNotes`) so validation does not re-read them on every launch. - The vector create handler goes through the modify debounce. - Drawings use a 30 s debounce in both indexers, so a drawing session costs one re-read once the user stops. - Fix the stale "10s" debounce comment (it is 5 s). Co-Authored-By: Claude <noreply@anthropic.com>
|
|
|
||
| // Without the data section it is not a drawing in the format we know; keep it all. | ||
| let content = match ? raw.substring(0, match.index) : raw; | ||
| content = content.replace(RE_LEADING_FRONTMATTER, "").replace(RE_EXCALIDRAW_BANNER, ""); |
There was a problem hiding this comment.
Existing unchanged drawings never get the new extraction rules. Both indexes accept saved entries when their mtime matches, and this PR leaves their saved versions unchanged. Blank .excalidraw.md drawings already indexed with the banner and frontmatter keep those search entries and vectors until an edit or manual rebuild. Add a one-time refresh for affected drawings and an upgrade test with an existing index.
Knowledge Base Used: Vector indexing and storage
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/utils/fileFiltering.ts
Line: 710
Comment:
**Old drawing entries remain**
Existing unchanged drawings never get the new extraction rules. Both indexes accept saved entries when their `mtime` matches, and this PR leaves their saved versions unchanged. Blank `.excalidraw.md` drawings already indexed with the banner and frontmatter keep those search entries and vectors until an edit or manual rebuild. Add a one-time refresh for affected drawings and an upgrade test with an existing index.
**Knowledge Base Used:** [Vector indexing and storage](https://app.greptile.com/your-papa/-/custom-context/knowledge-base/s2b-dev/smart-second-brain/-/docs/vector-indexing-and-storage.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Not changing this. Issue #541 comes from the per-autosave work: the full re-read, the re-embed and the create-time embed. That is fixed whether or not old entries are refreshed. A drawing already indexed with the banner keeps that one stale vector until its next edit, when it is re-read under the new rules (and dropped if blank), or until the user rebuilds the index. A one-time refresh would need a persisted migration marker that reaches into both indexes, for a cosmetic noise-floor vector per untouched drawing. That isn't worth the extra machinery in this fix.
There was a problem hiding this comment.
That’s a fair scope tradeoff. This PR fixes the repeated autosave re-read/re-embed path, while an untouched drawing’s legacy entry remains only until its next edit or an explicit rebuild. A persisted migration across both indexes would add substantial machinery for a noise-floor cleanup, so I’m withdrawing the finding.
…xt reads, batch empty-note saves Review follow-ups on #549: - Drawings keep their frontmatter in the extracted text (lexical search reads aliases from it); hasNoEmbeddableText looks past it instead. - An empty extraction from a PDF/.chat or a mobile size-gated file may be a failed read, so it is never remembered as empty. - Empty verdicts found by a bulk run are saved at checkpoints, not per note. Co-Authored-By: Claude <noreply@anthropic.com>
What
With S2B enabled, every Excalidraw autosave made both indexers re-read the whole drawing file (scene JSON included) on the main thread, and the vector index then embedded it as a title-only vector. The back-of-note extraction kept the frontmatter and Excalidraw's "Switch to EXCALIDRAW VIEW" banner, so a blank drawing never looked empty. New files were also embedded immediately on create, with no debounce. This is the likely cause of the iPad drawing breakage and the "couldn't embed" notice. S2B never writes to the drawing, so file locking was ruled out.
.excalidrawscenes are non-embeddable but still found by name in lexical search.emptyNotes, same lifecycle asfailedNotes, shown as "No text to embed" in the indexing report), so validation doesn't re-read every drawing on each launch.createhandler now goes through the modify debounce.Fixes #541
How I tested it
Unit tests: drawing extraction (blank, back-of-note text, older format),
.excalidrawembeddability, the empty-note verdict surviving a relaunch without re-reading and clearing on edit, an emptied note dropping its stale vectors, and the 30 s drawing debounce on both indexers.bun run check,format,lintandtestall pass. Not device-verified: Leo will test on iPad. A dev build is in slot wt2 (S2B WT2).AI assistance: Claude Code (Opus) wrote the change and the tests from Leo's brief to address #541. Leo reviews the change and tests it on iPad before merging.
Checklist
bun run check,bun run format,bun run lint, andbun run testpass locallymanifest.json: I noted that the docs site needs updating (n/a)