Skip to content

[FEAT] Task preflight capability check + verified-state grounding for agent claims - #80

Open
ScottieR wants to merge 3 commits into
masterfrom
cuj/preflight-capability-check
Open

ScottieR wants to merge 3 commits into
masterfrom
cuj/preflight-capability-check

Conversation

@ScottieR

Copy link
Copy Markdown
Owner

Fixes #60, fixes #53 — from the 2026-08-24 Sloane health-forms CUJ test, where the user learned only at minute 12 (by interrogating the agent) that email-sending and a working browser were never available, after the agent had claimed "everything is fully reconnected and ready now — I'm on it!"

Changes

  • Send-time preflight (detectTaskPreflightGaps): deterministic keyword capability match — email-send verbs without email_write, portal/website tasks with the browser off or not running, Slack/calendar mentions without the integration. Gaps render as a dismissible amber banner above the composer with Fix in Connections / Open Diagnostics buttons. Never blocks the send. Unit tests include the original CUJ message verbatim against Sloane's exact configuration.
  • Verified-state grounding (build_agent_health_context): every invocation's runtime context now carries a host-confirmed block — provider keys, browser RUNNING/NOT RUNNING, integrations with an explicit "email is READ-ONLY" call-out — plus an instruction to never claim readiness beyond it and to surface missing capabilities instead of promising to start.

Notes

Testing

  • 7 new preflight unit tests (incl. the real CUJ message); cargo check clean; tsc --noEmit clean; full frontend suite passes

🤖 Generated with Claude Code

… agent claims

From the 2026-08-24 Sloane CUJ test (issues #53, #60):

- detectTaskPreflightGaps(): deterministic keyword-based capability match
  run at send time. When a task mentions email-sending, portals/websites,
  Slack, or scheduling that the agent provably lacks (no email_write,
  browser off or not running, integration missing), a dismissible amber
  banner appears above the composer naming each gap with a one-click
  'Fix in Connections' / 'Open Diagnostics' path. Non-blocking — the
  send proceeds; the banner converts a 12-minute discovery-by-
  interrogation into a 10-second heads-up. Unit-tested against the
  original CUJ message verbatim.
- build_agent_health_context(): every invocation's runtime context now
  carries a host-verified state block (model provider keys, browser
  running or not, integrations incl. an explicit email READ-ONLY call-
  out) with an instruction not to claim readiness beyond it. The
  workspace DIAGNOSTICS.md injection stays, but it's only read if the
  agent chooses — this block is unmissable, which is what 'everything is
  fully reconnected, I'm on it!' over a dead browser demanded.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review (Claude)

**Verdict: ** | **Risk: **

Blocking issues:

improvement suggestion(s) filed as issues for later triage (see repo Issues, label claude-suggestion).

…vals

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant