Skip to content

Agentic guards: goal_met/prove_goal, hint origin, built-in rules for planner Bash (#124 F5, F6, F11) - #190

Merged
BaaRai merged 1 commit into
mainfrom
feat/agentic-guards
Nov 11, 2026
Merged

BaaRai merged 1 commit into
mainfrom
feat/agentic-guards

Conversation

@BaaRai

@BaaRai BaaRai commented Nov 11, 2026

Copy link
Copy Markdown
Owner

Phase B, PR 2 of 2 for #124: audit findings F5 (goal guards), F6 (truthful hint origin) and F11 (built-in deny rules for the planner and the main agent).

What and why

F5, goal guards (agent/tools.go)

The audit (goal drift, memory poisoning) found that goal_met ended a task without any check, and that prove_goal accepted any fact as evidence. A planted or inferred fact could therefore close a goal or the whole task.

  • goal_met lists the task's goal nodes. If any goal is not met, it refuses with the number of open goals and points to prove_goal; GoalMet is not set. If the goals cannot be read, it refuses. A task with no goal node keeps the old behavior. The tool now uses writeExpTool, so a ToolSet without an exploration store gets the standard "requires a task context" refusal instead of a nil dereference.
  • prove_goal with a fact reads confidence from the payload and requires exactly observed (trimmed, case-insensitive). Otherwise it refuses with a clear message and the goal stays open. Findings are still accepted as before.

F6, hint origin (agent/tools.go, agent/planner.go, agent/mainagent.go, server/manager.go, server/orchestration.go)

Every add_hint was stored with origin human, and the planner trigger said "The human (main agent)", including for hints written by the cross-task orchestrator.

  • addOneHint stores t.worker as the origin: human for the main-agent chat, orchestrator for cross-task orchestration, otherwise the agent name.
  • The origin is passed to the trigger: SetNotifyHint/notifyHint take (origin string, texts []string), Task.NotifyHint(origin, texts), and TriggerEvent gains HintOrigin. server/server.go needs no change (it passes the method value).
  • renderTriggers names the writer: "The human (via the main agent)", "A cross-task orchestration", "An agent (worker-N)", or "A source that was not recorded" when the origin is empty. The add_hint description no longer says the hint comes from the human or main agent.

F11, built-in rules for the planner and the main agent (intercept/intercept.go, guard/guard.go)

The planner and the main agent run Bash under NewTargetOnly, which applied no intercept rule at all.

  • New Interceptor.MatchBuiltin(tool, input): the loaded, enabled rules whose name starts with [built-in] (the v1/v2/v3 seeds) and whose action is deny, matched with the existing ruleMatches, first match in priority order. Without a database and without loaded rules it returns false, so store-less interceptors never touch a store.
  • guard.applyIntercept: for a target-only guard, MatchBuiltin runs first, before the target gate and before the early return when the gate is off. A match is logged as denied and blocked with the rule's message. User rules and the judge stay out of the target-only guard. The worker path is unchanged.
  • MatchBuiltin only considers the intercept-enabled tools (by default Bash, WebFetch, web_search, shell_open, shell_send, Write, Edit, MultiEdit), the same set as the worker path. Without this, a tool_input rule such as "[built-in] Delete-style endpoint paths" would deny add_hint or add_intent calls whose text just names a /delete endpoint.

Limits

  • Declared confidence. The fact's confidence is declared by the model that called record_fact. This PR does not check where the fact came from. The source check (source != target_response) comes with F4 (fact provenance) in 0.7.0.
  • Refusal, no pending proof. The audit allowed queueing an unproven fact for operator confirmation. In 0.6.0 there is no "pending proof" UI: prove_goal on a non-observed fact is a plain refusal and the goal stays open.
  • Test seam. Interceptor.SetRulesForTest(rules) installs rules (and the default enabled tools) on an interceptor without a database. The guard tests need it because they cannot reach the unexported rule cache. Interceptor.Log now returns early when there is no store, which is needed by the same store-less tests and changes nothing in production, where the store is always set.
  • Stored tool descriptions. Tool descriptions are seeded into the tools table only on first insert, and the model sees the stored text. The new add_hint and prove_goal descriptions only reach fresh databases. Existing installs keep the old text until the descriptions are refreshed. This PR does not bump tool_schema_refresh_*, because that would also overwrite edited descriptions of every orchestration and platform tool. The guards themselves are in the handlers and apply everywhere.
  • The target-only guard records a built-in deny in its own audit ring through block(), as it already does for target-gate blocks.

Tests

PG-unit (agent/agentic_guards_test.go, against a real Postgres):

  • TestGoalMetRefusesWhileGoalsOpen: an open goal plus goal_met returns an error naming "1 of 2 goals" and prove_goal, and GoalMet stays false. Once all goals are met, goal_met succeeds.
  • TestGoalMetWithoutGoalNodes: with no goal node, goal_met succeeds.
  • TestProveGoalRequiresObservedFact: an inferred fact and a fact without confidence are refused, and the goal stays open. An " Observed " fact marks the goal met. A finding marks the last goal met and completes the task.
  • TestAddHintRecordsOrigin: add_hint through an orchestrator ToolSet stores origin orchestrator and passes it to the trigger. Through the main agent's human ToolSet, it stays human.

Pure:

  • TestRenderTriggersHintOrigin: wording for human, orchestrator, worker-3, and an empty origin (single and batch), and never "The human (main agent)".
  • TestTargetOnlyGuardAppliesBuiltinDenyRules: a store-less interceptor with rules injected through SetRulesForTest. With the gate off and on, Bash rm -rf / is blocked by the built-in rule before the gate is reached, ls -la passes, and a user rule (no [built-in] prefix) is not applied. add_hint text naming a destructive command is not matched. The full guard still applies the user rule.
  • TestTargetOnlyGuardSkipsRules: unchanged assertions. An interceptor with no database and no loaded rules still passes every call without touching the store. The comment now says that in production the built-in rules apply.

Local gate (Kali, Go 1.26.7, throwaway postgres:16-alpine container):

  • go build ./...: OK
  • go vet ./...: OK
  • go test -count=1 -p 1 ./agent ./guard ./intercept ./server: all four ok
  • gofmt is clean for the lines this PR changes. gofmt -l still lists agent/tools.go (detectStagnation, stagnation_warning) and server/manager.go (the Task struct) for misformatting that predates this branch; I left it alone to keep the diff surgical.

Not verified

  • go test -race was not run locally (CI runs it on ./agent ./guard ./intercept).
  • Only ./agent ./guard ./intercept ./server were tested locally; other packages were only built and vetted.
  • No end-to-end run with a live LLM: the planner's reaction to the refusals and to the new trigger wording was not observed.
  • F11 against the real seeded rules was not tested in the guard package (the pure test uses an injected copy of the v2 "Destructive system commands" rule). The ./server and ./intercept suites, which load the real seeds, still pass.

Audit findings F5, F6 and F11 for issue #124.

F5, goal guards:
- goal_met refuses while any goal node of the task is open, and when the
  goals cannot be read. A task without goal nodes keeps the old behavior.
- prove_goal accepts a fact as evidence only when its confidence is
  "observed"; otherwise it refuses and the goal stays open. Findings are
  still accepted.

F6, hint origin:
- add_hint stores the calling ToolSet's worker as the hint origin instead
  of "human", and passes it to the planner trigger (TriggerEvent.HintOrigin).
- renderTriggers names the writer: the human via the main agent, a
  cross-task orchestration, or an agent, with a neutral fallback.

F11, built-in rules for the planner and the main agent:
- Interceptor.MatchBuiltin evaluates the enabled "[built-in]" deny rules on
  the intercept-enabled tools.
- The target-only guard applies them before the target gate, whether the
  gate is on or off. User rules and the judge stay out of it.
- SetRulesForTest is a test seam for the store-less guard tests; Log is a
  no-op without a store.
@BaaRai
BaaRai merged commit a1f3256 into main Nov 11, 2026
4 checks passed
@BaaRai BaaRai mentioned this pull request Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant