feat(prototype): check bound documents for leaks, and filter the queue by type - #124
Merged
AndrejK666 merged 1 commit intoSep 11, 2026
Conversation
…e by type
Two things from the queue.
## Leak, from the place that knows the type
A stage can gate on any detector, but only `purpose` was ever recorded against a
bound file, so a stage gating on `leak` could never complete from this flow.
`leak` belongs here for a reason the service enforces: it refuses a document
whose type it has not been told — 422, "pass doc_type, or a path naming the
type" — because "foreign content" means nothing until you have said what native
content would be. A bound document has a type. Nothing is guessed.
It runs over the documents that have one, and writes the verdict both places, as
the purpose run does: the finding to the graph, the pass/fail to the binding,
which is what the gate reads.
### It runs unverified, and that was measured
The service verifies its own candidates with an LLM by default, and on this
deployment that pass clears every one of them. Two documents of known kind,
declared correctly and incorrectly, unverified:
declared as passed share
a design contract design yes 0.00
a design contract adr NO 0.68
a design contract prd NO 1.00
a real ADR adr yes 0.00
a real ADR prd NO 0.99
a real ADR design NO 0.99
With verification on, all six pass — including the ADR read as a PRD at 0.99
raw. A check that never fires is not evidence that documents are clean, and a
gate wired to one would open for everything while looking like it had checked.
So the run passes `verify: false`, and the table is in the code.
That also gives the queue a second opinion on its own classification: a document
bound to the wrong type fails its leak check.
## Not bloat or traceability
They are set-wise: one run over a document set producing `duplicates` and
`traces_to` relations, with no per-document verdict anywhere in the response.
Gating a stage on them would mean inventing what "this document passed bloat"
means, so they stay in the Analyze tab, where relations are what they are.
## Filter by type
The queue filtered by state only. It now also filters by document type — the
types this project's documents actually are, not the whole catalogue, plus the
undetermined ones.
Signed-off-by: Andrej Kuchma <Andrej.Kuchma@constructor.tech>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two things from the Documents queue.
1. Leak, run from the place that knows the type
A stage can gate on any detector, but only
purposewas ever recorded against abound file — so a stage gating on
leakcould never complete from this flow.leakbelongs here for a reason the service itself enforces: it refuses adocument whose type it has not been told —
— because "foreign content" means nothing until you have said what native
content would be. A bound document has a type. Nothing is guessed, and nothing
is inferred from a filename.
It runs over the documents that have a type and writes the verdict both places,
as the purpose run does: the finding to the graph, the pass/fail to the binding,
which is what the gate reads.
It runs unverified, and that was measured
The service verifies its own candidates with an LLM by default, and on this
deployment that pass clears every one of them. Two documents of known kind,
declared correctly and incorrectly, unverified:
Right type → passes at 0.00. Wrong type → fails, and the share tracks how wrong.
With verification on, all six pass — including the ADR read as a PRD at 0.99
raw. Omitting
verifybehaves the same asverify: true, which is how thefirst few probes came back clean and nearly sent this out as a gate that always
opens.
A check that never fires is not evidence that documents are clean, and a gate
wired to one would open for everything while looking like it had checked. So the
run passes
verify: false, and the table lives in the comment ondetectLeakso nobody has to re-measure it.
That also gives the queue a second opinion on its own classification: a document
bound to the wrong type fails its leak check.
2. Not bloat or traceability
They are set-wise: one run over a document set producing
duplicatesandtraces_torelations, with no per-document verdict anywhere in the response(
materializeSetwiseturns them into graph edges, which is all they can be).Gating a stage on them would mean inventing what "this document passed bloat"
means. They stay in the Analyze tab, where relations are what they are.
3. Filter by type
The queue filtered by state only — needs review, bound, not documents, all. It
now also filters by document type: the types this project's documents actually
are, rather than the whole catalogue, plus the undetermined ones.
tsc -band the 41 prototype tests clean. Frontend only.