Skip to content

[Proposal] Per-turn model usage on the wire, so a Guardian can enforce cost ceilings #124

Description

@arsentev-ai

The problem

A Guardian cannot enforce a per-run or per-session cost ceiling, which OWASP LLM06:2026 Unbounded Consumption recommends in Mitigation #9 (Agentic Circuit Breakers), because no hook payload reports how much model inference a turn consumed. turnEnd has step_count and tool_call_count, and sessionEnd.summary has outcome, step_count and errors. The only token figure is preCompact.estimated_input_tokens.

Step counts are a weak proxy. In 722 measured coding-agent sessions (150,902 model calls), context processing was 83.5% of modelled cost, and sessions over 200 model calls were 8% of sessions but 92.2% of cost. Step cost grows with accumulated context (LLM06 Scenario #8), so turns with equal step counts can differ widely. Full data and sources: #123.

Why the wire has to carry it

Principle 3(a): the host receives per-call usage from the model provider, but that information cannot be reconstructed per ACS session_id, turn_id or subagent afterwards. Provider billing is aggregated per API key or project and arrives late. That is too late for a ceiling that must stop the next turnStart or toolCallRequest. The Guardian cannot estimate usage itself because it never sees the assembled prompt (#122).

Which constituencies this affects

  • Observed Agent implementers
  • Guardian implementers
  • Policy authors
  • Auditors and incident responders
  • Platform and harness vendors
  • Enterprise deployers

Alternatives considered

  1. Ride existing elements. turnEnd.summary is a provenance-carrying text summary, not telemetry. sessionEnd.summary could take optional totals, but only at session end, which is too late for a ceiling. policy_data carries Guardian intent toward the agent, not host-reported numbers.
  2. OTel traces only. This is fine for dashboards, but the Guardian in the decision path does not consume traces. The proposal reuses OTel names for mapping instead (gen_ai.usage.input_tokens, gen_ai.usage.cache_read.input_tokens, gen_ai.usage.output_tokens).
  3. A conformance profile ("cost accounting") instead of Core. This may be the right altitude, and it keeps the Core floor unchanged.

Sketch: an optional, audit-only usage object on turnEnd (model_calls, input_tokens, cache_read_input_tokens, output_tokens), with optional cumulative totals in sessionEnd.summary and subagentStop. No new MUST.

Prior art on a per-run resource record: IETF Internet-Draft draft-arsentev-agent-run-metrics (disclosure: I am its author, and of the measurements cited in #123).

Discussion link

#123

Current Priority Scope

This is deferred or out of scope and I am filing it to be tracked

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    status:needs-triageNot yet triaged. Applied by the issue formstype:proposalNew feature or request

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions