Repository navigation
feat(findings): optional counterevidence and confidence_rationale - #193
Merged
Merged
Conversation
Add two optional free-text fields to findings so a reporter can record what was tested and did not work (counterevidence) and why the conclusion holds (confidence_rationale). - db: idempotent ADD COLUMN IF NOT EXISTS, TEXT NOT NULL DEFAULT '', no CHECK; RecordFindingInput, the INSERT, the shared select list and all three scan sites carry them. - agent: report_finding accepts both as optional strings; the worker and pentest prompts say to fill them when confidence is not high. - server: FindingDTO (list and detail) carries both, omitted when empty; existing databases get the two schema properties added to the stored report_finding row once, without overwriting user edits. - web: the finding detail page shows each section only when non-empty. Both fields stay optional: a missing field never refuses a finding.
BaaRai
added a commit
that referenced
this pull request
Nov 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds two optional free-text fields to findings (Tier 1 item 3, finding evidence schema):
counterevidence: what was tested and did not work (blocked variants, negative paths).confidence_rationale: why the conclusion holds.Changes:
db/schema.sql:ALTER TABLE findings ADD COLUMN IF NOT EXISTS ... TEXT NOT NULL DEFAULT ''for both. Idempotent, no CHECK. Existing rows read back as''.db/finding_traffic.go:RecordFindingInputgets both fields, and theINSERTwrites them.db/findings.go:DBFindinggets both fields. They are added to the sharedfindingSelectColsand to all three scan sites (scanFindings,ListFindingsForExport,GetFinding).agent/tools.go:report_findingacceptscounterevidenceandconfidence_rationaleas optional string properties and passes them intoRecordFindingInput.agent/worker.go,agent/promptcatalog.go(pentest): one sentence on thereport_findingbullet. Fill both fields when confidence is not high. They are optional and never block reporting.server/dto.go:FindingDTOcarries both (omitempty). It is used by the list endpoints and by the detail endpointGET /api/exploration/findings/{id}.server/finding_verification.go+server/orchestration.go:seedFindingEvidenceSchema()adds the two properties to the storedreport_findingtool schema on existing databases. Without it, the model would never see them, becauseSeedToolis first-insert only and the DB schema overrides the code schema. It follows the same pattern asseedFindingVerificationSchema: a property is added only if it is missing, it is idempotent, and it preserves user edits to the description, other properties, bindings and the enabled flag.web/src/lib/types.tsandweb/src/app/(main)/function/findings/detail/page.tsx: the detail page shows a "Counterevidence" card and a "Confidence rationale" card, each only when its value is non-empty.Why
A confirmed finding currently records the positive proof only. Reviewers cannot see what the reporter already ruled out, or why the reporter believes the conclusion. That matters most when confidence is not high. These two fields give that reasoning a fixed place, outside the free-text evidence.
Optional fields, and why
Both fields are optional. A missing field never refuses a confirmed finding. This is the same philosophy as
traffic_refs: a valid finding must not be lost over a missing auxiliary field. The tool descriptions and the prompts ask for them when confidence is not high, but the tool does not enforce it.Limit
Other tools apply a rule like "required when confidence is not high". That rule cannot be enforced here, because findings have no confidence field.
report_findingcannot tell a low-confidence report from a high-confidence one, so the requirement stays as prompt guidance only. A structured confidence field, and enforcement based on it, can come in 0.7.0 if needed.Prompt seeds are first-insert only. Existing databases keep their stored worker/pentest prompt, so the new prompt sentence reaches new installs only. The tool-schema descriptions, which also carry the guidance, do reach existing databases through
seedFindingEvidenceSchema.Tests
agent/TestReportFindingSchemaReasoningFieldsOptional. Both properties exist, aretype: string, are described as optional, and are not inrequired.server/TestFindingDTOReasoningFields.findingFromDBmaps both fields, and the JSON omits them when they are empty.db/TestFindingReasoningFieldsRoundTrip. Stores a finding with and without the fields, then reads back viaGetFinding,ListFindingsPageandListFindingsForExport. Empty values default to''.agent/TestReportFindingStoresReasoningFields. Thereport_findingtool passes both fields into storage, and a call without them is still recorded.server/TestFindingReasoningDetailAndSchemaUpgrade. The detail endpoint returns both fields. The schema upgrade adds both properties to a customizedreport_findingrow (run twice) and does not overwrite the custom description, properties,requiredor enabled flag.Local verification (WSL Kali, Go 1.26.7, throwaway
postgres:16-alpinecontainer on 127.0.0.1):go build ./...: OKgo vet ./...: OKgo test -count=1 ./agent ./server ./dbon a fresh database:okfor all three packages. 642 top-level tests passed, 0 failed, 3 skipped: 2 inagent(known fresh-database skips that already exist onmain) and 1 inserver(live model config not set). All 5 new tests ran and passed.gofmt -l(CR stripped) on the changed files: onlyagent/tools.gois reported, for two hunks that already exist onmain(detectStagnationstruct alignment and thestagnation_warningmap). Neither is touched by this PR.npx tsc --noEmitexit 0.npm run buildexit 0.NEXT_EXPORT=1 next build(thebuild:staticcommand used in CI) exit 0.Not verified