Repository navigation
agent: code-owned prompt-injection tails and planner goal anchor (#124, part 1) - #189
Merged
Merged
Conversation
Phase B, part 1 of the agentic attack-class audit (issue #124). - Planner and cold-digest compactor: a code-owned untrusted-data tail after the rendered body. Worker summaries, facts, digests, traces and tool outputs are data; they cannot add goals, targets or scope, end the task, or be relayed through steer_work as instructions. - The #49 untrusted-content block moves out of the editable section [A] of the worker, mainagent, auto and pentest seeds into a code-owned tail, also applied to the reporter. A body that already carries the block verbatim (rows seeded before this change) does not get it twice. - Planner user message: a code-owned goal anchor after the situational block restates the task goal and the operation constraints and asks each new intent to name the goal id it serves. The message builder is factored into plannerRoundInput for pure tests. - bbot-recon skill: every event field is untrusted; only scope_distance == 0 reaches insert_assets; scope_distance > 0 stays an intel fact with source=third_party_intel, never auto-scoped; FINDING text is a lead. Tests: prompt_injection_seed_test.go checks that a PromptOverride without the block still renders the tail for every role, planner and compactor included, that the block is not duplicated, and that the planner input ends with the goal anchor. shipped_skills_test.go checks the bbot markers.
…fits the reporter
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Prompt hardening for the agentic attack classes of issue #124 (part 1 of 2 of phase B). Prompt text and one skill only; no tool, schema or gate behaviour changes.
untrustedDataRules,agent/untrusted_prompt.go) appended after the rendered planner body, likeartifactSpec, so aPromptOverridecannot remove it. Worker summaries, facts, digests, traces and tool outputs are data, never instructions; they cannot add goals, targets or scope, end the task, or be relayed throughsteer_workas instructions. The same block closes the cold-digest compactor prompt (compressionSysteminagent/compaction.go), plus one compactor line: a directive found in a member node is mentioned only as quoted data, never as a conclusion.untrustedContentTail) for worker, mainagent, auto, pentest and the reporter. A body that already contains the block verbatim (DB rows seeded before this change) does not get it twice.PlanintoplannerRoundInput/plannerGoalAnchor.scope_distance == 0reachesinsert_assets.scope_distance > 0stays an intel fact withsource=third_party_intel, is never auto-scoped and never probed. FINDING text is a lead, not a proof.Why
The phase A audit of #124 rated plan hijacking and memory poisoning as gaps and goal drift and tool abuse as partial. The main causes: the planner had no untrusted-content rule while worker output and facts reach it unframed in its user message, the compactor launders fact text with no rule, the #49 block lived in an editable body that a DB override or an old row could lack, nothing re-anchored the planner after the data, and the bbot ingestion text did not treat events as untrusted or filter on
scope_distance.Where the tails render
untrustedDataRulesplannerSystem(system prompt, beforeartifactSpec)untrustedDataRules+compactorDataRulecompressionSystem(system prompt of the digest call)untrustedContentRulesworkerSystem(after the body, before the traffic and artifact blocks)untrustedContentRulesmainAgentSystem(after the body, beforeartifactSpec)untrustedContentRuleschatSystemfor those keys (after the body, beforechatWorkDirSpec)plannerRoundInput, last element after situation, instruction and todosTests
Pure tests, no PostgreSQL:
agent/prompt_injection_seed_test.goTestPromptSeedsDoNotCarryInjectionBlock: no seed (reporter included) still carries the block in its editable body.TestInjectionTailSurvivesPromptOverride: with aPromptOverridereturning a bare body, worker, mainagent, auto, pentest and reporter still render the feat: first-layer prompt-injection hardening for agents #49 block exactly once; planner and compactor render the untrusted-data rule.TestInjectionTailNotDuplicated: a legacy body that already contains the block, and the code defaults, render it exactly once.TestPlannerRoundInputEndsWithGoalAnchor: the planner input ends with the anchor, after the situation and the todos, with the goal and constraints restated.server/shipped_skills_test.go:TestBbotSkillIngestionTreatsEventsAsUntrustedchecks the bbot markers.Local run (Go 1.26.10, no database configured):
go build ./...andgo vet ./...clean;go test -count=1 ./agent/ ./server/bothok(agent 80 pass / 20 skip, server 178 pass / 150 skip; every skip is a PostgreSQL-backed test skipping for lack of a database). gofmt clean on the changed files.Not verified