Skip to content

agent: code-owned prompt-injection tails and planner goal anchor (#124, part 1) - #189

Merged
BaaRai merged 2 commits into
mainfrom
feat/agentic-prompt-hardening
Nov 11, 2026
Merged

BaaRai merged 2 commits into
mainfrom
feat/agentic-prompt-hardening

Conversation

@BaaRai

@BaaRai BaaRai commented Nov 11, 2026

Copy link
Copy Markdown
Owner

What

Prompt hardening for the agentic attack classes of issue #124 (part 1 of 2 of phase B). Prompt text and one skill only; no tool, schema or gate behaviour changes.

  1. Planner and compactor untrusted-data tail (F1). A code-owned block (untrustedDataRules, agent/untrusted_prompt.go) appended after the rendered planner body, like artifactSpec, so a PromptOverride cannot remove it. Worker summaries, facts, digests, traces and tool outputs are data, never instructions; they cannot add goals, targets or scope, end the task, or be relayed through steer_work as instructions. The same block closes the cold-digest compactor prompt (compressionSystem in agent/compaction.go), plus one compactor line: a directive found in a member node is mentioned only as quoted data, never as a conclusion.
  2. feat: first-layer prompt-injection hardening for agents #49 block becomes a code-owned tail (F2). The untrusted-content block leaves the editable section [A] of the worker, mainagent, auto and pentest seeds and is appended by code (untrustedContentTail) for worker, mainagent, auto, pentest and the reporter. A body that already contains the block verbatim (DB rows seeded before this change) does not get it twice.
  3. Planner goal anchor (F7). The planner user message now ends with a code-owned anchor placed after the situational block and the todos: it restates the task goal and the operation constraints, and asks each new intent to name the goal id it serves; data cannot add goals, targets or scope. The message builder is factored out of Plan into plannerRoundInput / plannerGoalAnchor.
  4. bbot-recon skill (F10). Every event field is untrusted. Only scope_distance == 0 reaches insert_assets. scope_distance > 0 stays an intel fact with source=third_party_intel, is never auto-scoped and never probed. FINDING text is a lead, not a proof.

Why

The phase A audit of #124 rated plan hijacking and memory poisoning as gaps and goal drift and tool abuse as partial. The main causes: the planner had no untrusted-content rule while worker output and facts reach it unframed in its user message, the compactor launders fact text with no rule, the #49 block lived in an editable body that a DB override or an old row could lack, nothing re-anchored the planner after the data, and the bbot ingestion text did not treat events as untrusted or filter on scope_distance.

Where the tails render

Role Tail Rendered by
planner untrustedDataRules plannerSystem (system prompt, before artifactSpec)
compactor untrustedDataRules + compactorDataRule compressionSystem (system prompt of the digest call)
worker untrustedContentRules workerSystem (after the body, before the traffic and artifact blocks)
mainagent untrustedContentRules mainAgentSystem (after the body, before artifactSpec)
auto, pentest, reporter untrustedContentRules chatSystem for those keys (after the body, before chatWorkDirSpec)
planner user message goal anchor plannerRoundInput, last element after situation, instruction and todos

Tests

Pure tests, no PostgreSQL:

  • agent/prompt_injection_seed_test.go
    • TestPromptSeedsDoNotCarryInjectionBlock: no seed (reporter included) still carries the block in its editable body.
    • TestInjectionTailSurvivesPromptOverride: with a PromptOverride returning a bare body, worker, mainagent, auto, pentest and reporter still render the feat: first-layer prompt-injection hardening for agents #49 block exactly once; planner and compactor render the untrusted-data rule.
    • TestInjectionTailNotDuplicated: a legacy body that already contains the block, and the code defaults, render it exactly once.
    • TestPlannerRoundInputEndsWithGoalAnchor: the planner input ends with the anchor, after the situation and the todos, with the goal and constraints restated.
  • server/shipped_skills_test.go: TestBbotSkillIngestionTreatsEventsAsUntrusted checks the bbot markers.

Local run (Go 1.26.10, no database configured): go build ./... and go vet ./... clean; go test -count=1 ./agent/ ./server/ both ok (agent 80 pass / 20 skip, server 178 pass / 150 skip; every skip is a PostgreSQL-backed test skipping for lack of a database). gofmt clean on the changed files.

Not verified

  • No live run: no task was executed against the stack or a lab target.
  • No model behaviour was measured. Whether the planner, workers or compactor actually ignore instruction-shaped data, and the token cost of the new tails, is for the phase C lab.
  • The PostgreSQL-backed tests did not run locally; CI runs them.
  • Not in this PR: framing of agent-derived data in user messages (F3), fact provenance (F4), goal guards (F5), hint origin (F6), auto-scope hygiene (F8), the boundary default (F9).

Phase B, part 1 of the agentic attack-class audit (issue #124).

- Planner and cold-digest compactor: a code-owned untrusted-data tail
  after the rendered body. Worker summaries, facts, digests, traces and
  tool outputs are data; they cannot add goals, targets or scope, end the
  task, or be relayed through steer_work as instructions.
- The #49 untrusted-content block moves out of the editable section [A]
  of the worker, mainagent, auto and pentest seeds into a code-owned tail,
  also applied to the reporter. A body that already carries the block
  verbatim (rows seeded before this change) does not get it twice.
- Planner user message: a code-owned goal anchor after the situational
  block restates the task goal and the operation constraints and asks each
  new intent to name the goal id it serves. The message builder is
  factored into plannerRoundInput for pure tests.
- bbot-recon skill: every event field is untrusted; only scope_distance
  == 0 reaches insert_assets; scope_distance > 0 stays an intel fact with
  source=third_party_intel, never auto-scoped; FINDING text is a lead.

Tests: prompt_injection_seed_test.go checks that a PromptOverride without
the block still renders the tail for every role, planner and compactor
included, that the block is not duplicated, and that the planner input
ends with the goal anchor. shipped_skills_test.go checks the bbot markers.
@BaaRai
BaaRai merged commit 2945f58 into main Nov 11, 2026
3 checks passed
@BaaRai BaaRai mentioned this pull request Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant