Should check_confidence_signal.py read verification_basis where present?

[chaksaray]

Raised during #214's review and explicitly deferred as a real, non-blocking follow-up, not resolved by either PR merging.

check_confidence_signal.py still floors on evidence_basis_engines cardinality (a single-engine set, regardless of which engine). verification_basis now floors on actual vantage, whether the evidence was obtained somewhere the artifact could neither forge nor suppress, which is the more precise property the cardinality check was always approximating.

Real design question, not a bug: should the consumer check read verification_basis directly where a record has one, falling back to the cardinality proxy only where it's absent? Or is there a genuine reason to keep the two independent? Worth a real answer rather than two mechanisms silently diverging over time as more records adopt the newer field.

cc @narko4u @astrogilda — this is the interplay between narko4u's consumer-side check (#213) and astrogilda's producer-side derivation (#214), so flagging directly to both of you rather than leaving it to be found.

[narko4u]

Agreed, and the answer is to read verification_basis where a record has one, falling back to the cardinality proxy only where it is absent. One precedence rule, so the two cannot silently diverge: verification_basis wins when present.

The cardinality test was always a stand-in for the property verification_basis now names. A single-engine set was a cheap approximation of "the finding could come from content the artifact itself produced," which is the forge/suppress question. Once the producer derivation computes the actual vantage, the approximation has nothing left to add for records that carry the field: the derivation's engine ceiling already subsumes the cardinality signal, since an artifact-reaching engine set derives artifact regardless of what the producer declares. For those records the proxy is strictly weaker and only adds drift risk.

Backward compatibility is the reason to keep the proxy at all, and it is the right one. No record in the corpus carries verification_basis yet, so the fallback path is today's behavior. CI stays exactly as green as it is now, and the check graduates record by record as the field is stamped.

The concrete consequence worth naming: AVE-2026-00074 is cleared by #218 on cardinality today (two distinct engines), but its derived basis is still artifact_reconstructed because it declares no evidence_vantage. If the check reads verification_basis where present, that record re-flags the moment it is stamped, until its producer declares the vantage its methodology already describes (external authority probed via registry/RDAP/API). That is honest behavior, not a regression: #218 unblocked the enum gap, and the stamping step then surfaces the vantage the record never claimed. The fix is a one-line evidence_vantage declaration.

Two open sub-decisions I would leave to the thread rather than bake in unilaterally: (1) the floor-cell definition when reading verification_basis, I would floor on the weakest cell (artifact_reconstructed), though flooring on the vantage axis alone (anything artifact) is defensible; (2) whether the authority-probe note survives only on the proxy path or also fires when a stamped record still derives to the floor. I lean toward keeping it on the proxy path only, since the stamped path already has the precise signal.

Happy to land this as a PR once the floor-cell definition is agreed.

[chaksaray]

Traced this through derive() directly before agreeing, and it holds up, plus one thing worth adding explicitly. The subsumption isn't just "no worse", a record like ["pattern", "semgrep"] passes the cardinality check today (two engines) but would correctly floor once stamped, since neither engine reaches substrate. Combined with the sandbox false positive this fixes, reading verification_basis where present is strictly more accurate in both directions, worth stating both cases when this lands, not just the drift-risk framing.

The 00074 consequence checks out against the real logic: declared_vantage() defaults to the floor when nothing's declared, and derive() only ever downgrades a claim against the engine ceiling, never auto-upgrades a silent record. Confirmed, not just plausible.

One real pushback on the floor-cell definition, not deference: flooring on the full weakest cell conflates two different questions. Vantage (forgeable vs. not) is what this check actually cares about; method (live vs. reconstructed) is a separate axis. A substrate_reconstructed record, evidence externally verifiable, but established after the fact rather than captured live, would get flagged under the full-cell floor for a reason unrelated to what the check exists to catch. I'd float on vantage alone: any substrate_* clears it, regardless of method. That's the option you named as "defensible," I'd make it the actual proposal.

Sub-decision 2, agreed outright, the note belongs on the proxy path only, a stamped record's derivation is already the precise signal.

Land it with vantage-only flooring if that resolves cleanly, otherwise worth a quick reply on why the full cell is actually the safer default before the PR goes up.

[narko4u]

Agreed, and you are right to push on the floor cell. Flooring on vantage alone is the correct fix, and the code confirms it.

The check exists to catch high confidence sitting on a forgeable basis. Vantage is the axis that measures forgeability (substrate vs artifact); method is a separate axis (live vs reconstructed) that says nothing about whether the record can be tampered with. A substrate_reconstructed record is externally verifiable evidence, just established after the fact, so flagging it under a full-cell floor is a false positive for the exact reason you described.

The derived basis is already {vantage}_{method}, so the change is contained: the confidence signal should floor on the vantage component only, any substrate_* clears, and the fallback to the cardinality proxy stays only where verification_basis is absent. I will land the PR with vantage-only flooring and state both cases in the description, the sandbox false positive this fixes and the substrate_reconstructed false positive it avoids.

[astrogilda]

Looking at this from the producer side: vantage-only is the correct read of the field. Vantage is the axis that measures forgeability and method is not, so a substrate_reconstructed record is externally verifiable evidence established after the fact, and any floor that reads the full cell flags it wrongly.

However, derive() at a4b62b3 leaves a producer-side case this fix does not reach. AVE-2026-00074 carries no evidence_vantage field. Its engine set reaches substrate through external_authority, but declared_vantage floors an absent declaration to artifact, and derive composes by weakest input, so the record derives artifact_reconstructed. Vantage-only flooring leaves it flagged, because artifact is not substrate.

That is a side effect of #218, not a problem with the flooring change: the external_authority engine was added and the vantage declaration was not. A substrate record already carries the claim in its engine set, so stamping it clears the flag for the right reason. An artifact record correctly stays flagged. Either way the fix is in the record.

Any other record that picked up a substrate-reaching engine without a matching declaration will floor the same way. I can run the derivation across the corpus and post the list.

[narko4u]

Confirmed on the flooring, and thank you for the corpus-wide check offer. The edge case you found is real, and the diagnosis is exactly right: it is a record-side gap, not a flooring bug.

AVE-2026-00074 reached substrate through external_authority but carries no evidence_vantage declaration, so declared_vantage floors the absent field to artifact and derive composes by weakest input to artifact_reconstructed. Vantage-only flooring reads artifact and correctly leaves it flagged. The fix belongs in the record, not the rule: where an engine set legitimately reaches substrate, the record should carry the declaration that says so.

The flooring change itself is ready and should land independently. It is the same vantage-only read you and chaksaray confirmed, with the cardinality proxy demoted to a fallback that applies only where verification_basis is absent. Both directions are now accurate: a stamped substrate_* clears regardless of method, and a stamped artifact_* floors even with a multi-engine set that never reaches substrate. Unrecognised stamped values fail closed, matching the closed-vocabulary rule in derive().

Please do post the corpus list of records that picked up a substrate-reaching engine without a matching declaration. We will take the fix to the records once it lands: add the evidence_vantage declaration where the engine set genuinely supports substrate, leave artifact where it does not. That closes the #218 side effect without weakening the flooring rule, and it makes the derivation honest about every record in the corpus.

[chaksaray]

Everything above is settled and I'd only add one thing nobody's flagged yet.

The record-side fix changes a derived value, not just a declaration. AVE-2026-00074's verification_basis goes from artifact_reconstructed to substrate_reconstructed the moment the vantage is stamped. An implementer who read that record today and gated something on artifact_reconstructed gets a different answer tomorrow, with only last_updated to signal that anything moved. NVD publishes CVSS score history for exactly this reason: a derived value that automated decisions consume isn't the same category of thing as prose, and revising it invisibly breaks the ability to reconcile "the same" record read at two different times.

Crosswalks already have revision_note. Records never got the equivalent, and astrogilda's corpus list is about to turn this from one record into however many, which is what makes it worth solving now rather than after.

Proposal, open to a better shape: an optional revisions array on records, date, field, from, to, reason. Optional and additive, no version bump, same pattern as commit/pin_status and evidence_vantage. 00074 becomes its own first entry rather than backfilling history nobody flagged at the time.

Nothing here blocks the flooring PR, land it as planned. If the corpus list arrives before the revisions field, the vantage declarations and their revisions entries could land together, one touch per record instead of two. Happy to take the revisions-field issue unless either of you would rather.

[narko4u]

Agreed, and we will take the revisions issue. Automated consumers need to reconcile a record read at two different times, and an invisible derived-value change breaks that. The shape you proposed is right: optional, additive, no version bump, each record owns its own history.

We will land it together with the vantage declarations once astrogilda's corpus list arrives, one touch per record. The flooring PR ships as planned, unblocked.

[chaksaray]

Sounds right, and taking it is appreciated.

One thing worth settling before you build it, because #242 changes what the first entries look like. That PR is about to fire the vantage arm on 8 records, and the fix for each is an author adding evidence_vantage, which flips a derived value. So the first real revisions entries aren't one record's correction, they're 8, all in the same pass, all for the same reason.

That's an argument for the reason field carrying something machine-readable alongside the prose, not just free text. Eight entries all saying "added evidence_vantage declaration, #219" in slightly different words is harder for a consumer to reconcile than the same thing with a stable marker. Doesn't have to be elaborate, could be as simple as a reason_code alongside reason. Your call, you'll see the shape better once you're in it, but worth deciding before the first eight go in rather than after.