Skip to content

agent: fact provenance (declared source plus server-set origin) - #194

Merged
BaaRai merged 2 commits into
mainfrom
feat/fact-provenance
Nov 12, 2026
Merged

BaaRai merged 2 commits into
mainfrom
feat/fact-provenance

Conversation

@BaaRai

@BaaRai BaaRai commented Nov 11, 2026 •

Copy link
Copy Markdown
Owner

What

F4 from the agentic attack-class audit (#124): fact provenance, for 0.7.0.

  • record_fact takes a source field from a closed enum: target_response, tool_output, third_party_intel, operator_input, inference. An empty value is stored as inference. Any other value is refused with a message that names the value and the allowed list. In batch mode the bad item is reported in errors and the other items are written. The source is stored in the fact payload.
  • The writer of a node is already set by the server in the origin column (worker-N, planner, human, orchestrator). It is now surfaced. It is read only from that column, never from model text.
  • compactNode carries origin and source, so they appear in recent_facts (overview, including related-task facts), list_facts, and expand_digest members. The cold-digest compressor input carries source and origin next to confidence. The digest body is still model text; only its input changes.
  • prove_goal refuses a fact whose source is target_response, in addition to the existing confidence=observed rule. The message is explicit and the goal stays open. Findings are still accepted.
  • The default worker, pentest and planner prompts ask for the source when recording facts. The planner treats target_response as non-authoritative for proving a goal.
  • On an existing database, seedFactSourceSchema (run from seedOrchestrationTools) adds the source property with its enum to the stored record_fact schema. SeedTool is first-insert only and ToolResolve serves the stored row, so without this the model never saw the enum and every fact went to inference. The update only adds the missing property: operator edits to the description and other properties are kept, and a second run changes nothing.

Why

The audit found a memory-poisoning gap and partial goal drift. A fact stored only the writing agent's name, which never reached a prompt. Nothing said whether its content came from the target. A worker could transcribe target text into a fact, and that fact then reached the planner overview, cold digests and prove_goal. A target could therefore plant a statement that closes a goal. This PR records where the content came from and shows it to the readers that act on facts. It also stops prove_goal from accepting target-sourced facts.

Known limits

  • Inferred facts stay confirmed. Every fact is still stored with state confirmed, whatever its confidence. Changing that is a separate behaviour change that should be measured first; it is not in this PR.
  • The source is model-declared. Only origin is server-set. A model can mislabel target content as tool_output or inference, and the server cannot check that. This PR makes the declaration explicit and visible; it does not verify it.
  • target_response is only refused by prove_goal. Other decisions do not filter on it: add_intent parents, worker mandates, cold digests and the related-task overview. The planner prompt treats it as non-authoritative for goals only.
  • Facts written before this change have no source. They show no source key and are not refused by the new prove_goal rule (the confidence rule still applies).
  • The prompt changes are in the code defaults. Like earlier prompt PRs, there is no reseed flag, so a database that already stores a prompt version keeps it until it is reset to the default. The record_fact and prove_goal tool descriptions carry the rule on a fresh database; see the next point for existing databases.
  • Stored tool descriptions on existing databases. The stored descriptions of prove_goal and list_facts stay old on an existing database. That is acceptable: the refusals are enforced server-side, whatever the description says. The record_fact schema was the only blocking piece (without the source property the model could not declare a source), and it is now upgraded by seedFactSourceSchema. The stored record_fact description also stays old; the added source property carries its own description and enum.
  • The compressor system prompt still describes its node lines as carrying confidence. It is not changed here, so the model sees source= and origin= without a description.

Tests

New file agent/fact_provenance_test.go.

Pure (no database):

  • TestNormalizeFactSource: empty becomes inference, every enum value is kept, case and spaces are ignored, invalid values are refused with the value and the allowed list.
  • TestCompactNodeProvenance: origin and source are surfaced; a legacy node carries neither key.
  • TestCompressionInputProvenance: the digest input line carries confidence, source and origin; a legacy line is unchanged.
  • TestPromptSeedsCarryFactSource: the worker and pentest seeds name every source; the planner seed treats target_response as non-authoritative.

PostgreSQL:

  • TestRecordFactStoresProvenance: source and origin are stored, the default is inference, an invalid value is refused with a clear message in single and batch mode, and nothing is written for it.
  • TestFactProvenanceSurfaced: recent_facts, list_facts, expand_digest members and the digest input built from the stored graph expose both fields.
  • TestProveGoalRefusesTargetResponseFact: an observed target_response fact is refused and the goal stays open; an observed tool_output fact proves; a finding proves.
  • TestRecordFactSourceSchemaUpgrade (server/fact_source_schema_test.go): the stored record_fact row is edited to an old shape (no source, custom description and parameter text), seedOrchestrationTools runs twice, and the test checks that source is added with the five enum values, that the operator text and other properties are intact, that the second run leaves the row unchanged (including updated_at), and that ToolResolve serves the source property to the worker. It fails when the seedFactSourceSchema call is removed.

Local gate (Go 1.26.10 linux/amd64, throwaway postgres:16-alpine container, fresh database):

  • go build ./...: ok
  • go vet ./...: ok
  • go test -count=1 -p 1 ./agent ./server ./db: ok for all three packages. The PG tests above ran and passed.
  • After the schema upgrade commit (Go 1.26.7 linux/amd64, fresh throwaway database): go build ./... ok, go vet ./... ok, go test -count=1 -p 1 ./server ./agent ./db ok for all three packages.
  • gofmt: every changed hunk is formatted. agent/tools.go already had unformatted lines on main (detectStagnation, stagnation_warning); they are left as they are.

Not verified

  • No live run with a model: whether models declare the source accurately is not measured.
  • go test -race and the full ./... run were not run locally; CI covers them.
  • The behaviour on a database whose prompts were customised was not checked.

record_fact takes a source field from a closed enum (target_response,
tool_output, third_party_intel, operator_input, inference). An empty value
is stored as inference; any other value is refused with the allowed list.
The source is stored in the fact payload.

The writer of a node is already recorded by the server in the origin
column. compactNode now surfaces origin and source, so recent_facts in the
overview, list_facts, expand_digest members and related-task facts carry
them. The cold-digest compressor input carries source and origin next to
confidence; the digest body is unchanged model text.

prove_goal refuses a fact whose source is target_response, in addition to
the existing confidence=observed rule. The goal stays open; findings are
still accepted.

The worker, pentest and planner default prompts ask for the source, and the
planner treats target_response facts as non-authoritative for goals.

Refs #124 (F4).
SeedTool is first-insert only and ToolResolve serves the stored tools row,
so on an existing database the record_fact schema never gained the source
property: the model never saw the enum, every fact was stored as inference,
and prove_goal's target_response refusal could not fire.

seedFactSourceSchema copies properties.source from the record_fact seed into
the stored schema, only when the stored schema lacks it (same pattern as
seedFindingEvidenceSchema). Operator edits to the description and to other
properties are preserved, and a second run changes nothing. It runs from
seedOrchestrationTools next to the other schema seeders.

The PG test edits the stored row (no source, custom description and
parameter text), runs seedOrchestrationTools twice, and checks the added
enum, the preserved operator text, the unchanged row on the second run,
and that ToolResolve serves the source property to the worker.

Refs #124 (F4).
@BaaRai
BaaRai merged commit ff75bfa into main Nov 12, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant