Skip to content

feat(http): count capability denials on the egress gate - #377

Merged
mfw78 merged 3 commits into
mainfrom
feat/count-capability-denials
Aug 26, 2026
Merged

mfw78 merged 3 commits into
mainfrom
feat/count-capability-denials

Conversation

@mfw78

@mfw78 mfw78 commented Aug 26, 2026 •

Copy link
Copy Markdown
Contributor

What

nexum_runtime_capability_denials_total, a counter labelled capability, reason and module. capability = "http" is the seam, so the axis scales when the extension seam adds another. reason separates the two refusals: allowlist, where the operator or the manifest did not name the host, and destination, where the address resolved into denied space.

Both emit sites are in crates/nexum-runtime-http/src/http.rs. The allowlist refusal counts in HttpGate::send_request; the address refusal counts at the reject_prohibited_destination call site inside send_with_limits, which now carries the module through. reject_prohibited_destination itself stays a pure predicate.

The destination refusal also gained a warn!, which previously logged nothing at all. It carries the module and the host only, matching the rule the allowlist line already followed about paths and query strings.

docs/production.md gets the metric row, a NexumCapabilityDenied alert, and a line under the policy-refusal table tying HttpRequestDenied and DestinationIpProhibited to the two reasons an operator will see in the counter.

Why

Closes #369

Nothing counted an egress denial, so a guest probing the gate raised nothing an operator could alert on. Folded in from #128, which asked for the operator denylist that landed without the counting.

Cardinality, and where each label comes from

No label carries a value a guest chooses. capability and reason are fixed strings from a Refusal enum. module is the operator-written id from engine.toml. A denied host is exactly the value an attacker controls and is deliberately absent: it appears in the log line, which is bounded, not in a label, which is not. That is the failure #308 was filed for.

reason comes from strum::IntoStaticStr rather than a hand-written match, so a third refusal cannot be added without its label.

What is not counted, deliberately

A transport failure is not a denial. Review found the first revision counted any send_request error as an allowlist refusal; it now counts only ErrorCode::HttpRequestDenied. A DNS failure or a connection reset would otherwise have inflated a security signal with ordinary network noise, which is worse than not counting at all. There is a test for it.

A hostless URI is not a denial either, for the same reason, and also has a test.

The chain read-surface denial is not included. The issue says count every capability denial and the chain gate is one, so this is a scope call rather than an oversight: nexum_runtime_chain_request_total already counts it as method="<denied>" with outcome="err", and #376 just gave those log lines their module. Adding a second counter for the same event would give an operator two numbers for one thing. If the chain gate should move under the shared metric, that is a deliberate consolidation and its own change.

The alert is for: 10m over a [5m] rate, so a single one-off refusal decays out of the window before it fires and only a module that keeps asking raises it.

The capture harness moved, so this does not add a sixth far consumer

The first revision reached capture_metrics through nexum-runtime-testing, which normal-depends on the chain, wasm and world crates. That took cargo nextest run -p nexum-runtime-http from 195 packages to 364, to assert on a counter, using only Sample, capture_metrics, samples_named and block_on_current_thread, none of which need any of it.

The harness now lives in nexum-runtime-metrics behind a testing feature. That crate is a true leaf with exactly one dependency, so any crate can reach it for free, and the capture belongs beside the table it captures. nexum-runtime-testing re-exports it, so no existing consumer changes, and it drops metrics from its own dependencies since the module that used it left.

nexum-runtime-http is back to 214 packages: metrics-util, tokio's rt, and the leaf crate itself.

This is the shape that also answers #364. A harness belongs with the subsystem it observes, which leaves nexum-runtime-testing as the composed mocks that genuinely need the whole engine rather than a bundle where every consumer pays for the heaviest member.

Testing

cargo nextest run -p nexum-runtime-http -p nexum-runtime-metrics -p nexum-runtime-guards -p nexum-runtime --all-features --locked: 117 passed. The guard in nexum-runtime-guards covers the new METRICS entry, so the table row and the emit had to land in one commit.

Four new tests drive real denials through the gate and assert the labels: an_allowlist_denial_is_counted_against_the_module, a_destination_denial_is_counted_against_the_module, a_transport_failure_is_not_counted_as_a_denial, a_hostless_uri_is_not_counted_as_a_denial.

Workspace clippy, rustdoc, doctests, fmt, machete, crate-lints, workspace-deps, zero-leak and content all clean. Cargo.lock gains three lines matching the three manifest additions, no version churn.

AI Assistance

Implementation: claude-opus-5. Red-team review: claude-opus-5. Verification: claude-opus-5. PR description: claude-opus-5.

mfw78 added 2 commits August 26, 2026 04:24
An allowlist refusal and a destination refusal now increment
nexum_runtime_capability_denials_total, labelled by capability, reason
and module, so a guest probing the gate raises something an operator can
alert on rather than a log line nothing watches. One name carrying a
capability label scales to the extension seam; a name per capability
does not.

The destination refusal runs inside the spawned send task, which did not
carry the module id, so send_with_limits takes it now. Every label is
fixed or operator-written: a host and a URL are guest-chosen, and either
would let a module mint series at will.

Adds the name to METRICS, the row to docs/production.md, and a
NexumCapabilityDenied rule that a single refusal cannot hold.

Closes #369

AI Assistance: Claude Code used for the implementation, the tests, and the docs.
`admit` refuses a hostless URI with `HttpRequestUriInvalid`, which is a
malformed request rather than a host the policy excludes. Counting it under
`reason="allowlist"` contradicted the row `docs/production.md` adds for the
series, and sent an operator to look for a manifest or `http_allow` entry that
was never involved.

Tighten `assert_one_denial` to the counter value as well as the label set: the
debugging recorder collapses repeated increments of one key into a single
sample, so the series check alone passed a double count.

AI Assistance: Claude Code used for the red-team review and this fix.
nexum-runtime-http reached capture_metrics through nexum-runtime-testing,
which normal-depends on the chain, wasm and world crates, so a near-leaf
crate's test build went from 195 packages to 364 to assert on a counter.
It used only Sample, capture_metrics, samples_named and
block_on_current_thread, none of which need any of that.

The harness moves to nexum-runtime-metrics behind a testing feature. That
crate is a true leaf with one dependency, so every consumer reaches it for
free, and the capture belongs with the table it captures.
nexum-runtime-testing re-exports it, so no existing consumer changes.

http is back to 214 packages: metrics-util, tokio's rt and the leaf crate
itself.

The same shape answers the log-capture sink question: a harness belongs
with the subsystem it observes, leaving nexum-runtime-testing as the
composed mocks that genuinely need the whole engine.

AI Assistance: claude-opus-5 used for the move and the measurement.
@mfw78
mfw78 merged commit fed4f6f into main Aug 26, 2026
7 checks passed
@mfw78
mfw78 deleted the feat/count-capability-denials branch August 26, 2026 07:17
mfw78 added a commit that referenced this pull request Aug 26, 2026
…e install

`nexum-runtime-testing` re-exported `LogCapture` and `capture_logs` with no
consumer. Reaching the sink through that crate costs alloy, tower and wasmtime
in a test build, which is the reach #377 removed for `capture_metrics`; every
call site in this change takes `nexum-runtime-logs` directly, so the re-export
is a cheap-looking path to an expensive dependency and nothing else. Its
`testing` feature on the normal dependency and the description edit go with it.

`LogCapture::install` returns a `DefaultGuard`. `tracing::subscriber::set_default`
is `#[must_use]`, but that does not carry through the wrapper, so
`sink.install(Level::INFO);` as a statement dropped the guard immediately and
captured nothing while the test still compiled. The attribute now sits on
`install`.

The comment above the `install` call in `harness.rs` restated the method's own
rustdoc and the `expect` string two lines below it.

AI Assistance: claude-opus-5 used for red-team review of the branch and these fixes.
mfw78 added a commit that referenced this pull request Aug 26, 2026
)

* refactor(testing): share one log-capture sink from the logs crate

The `io::Write` plus `MakeWriter` pair that reads back what `tracing` emitted was copied into five crates. `LogCapture` and `capture_logs` now live in `nexum-runtime-logs` behind a `testing` feature, beside the pipeline whose output they read, and `nexum-runtime-testing` re-exports them so a composed consumer reaches them under one name.

Four copies are gone: the logs crate's own `Console`, the wasm fault funnel's `Sink`, the facade harness's `LogSink`, and the supervisor event loop's `LogSink`. `nexum-tasks` keeps its local copy deliberately: it is layer 0 with a 42-package test graph, and the logs crate normal-depends on wasmtime-wasi, so sharing there would cost 216 packages to save 25 lines.

Every existing assertion is unchanged; the capture level each call site used is now an argument rather than a constant baked into its private copy. The shared sink carries three tests of its own: the level ceiling, the absence of ANSI escapes, and the scope of the install guard.

`tracing-subscriber` moves from a dev-dependency to an optional dependency on the logs crate, and drops out of the wasm and supervisor dev-dependencies along with the wasm crate's `parking_lot`.

Refs #364

AI Assistance: claude-opus-5 used for implementation, verification, and this commit message.

* refactor(testing): drop the unused log-capture re-export and guard the install

`nexum-runtime-testing` re-exported `LogCapture` and `capture_logs` with no
consumer. Reaching the sink through that crate costs alloy, tower and wasmtime
in a test build, which is the reach #377 removed for `capture_metrics`; every
call site in this change takes `nexum-runtime-logs` directly, so the re-export
is a cheap-looking path to an expensive dependency and nothing else. Its
`testing` feature on the normal dependency and the description edit go with it.

`LogCapture::install` returns a `DefaultGuard`. `tracing::subscriber::set_default`
is `#[must_use]`, but that does not carry through the wrapper, so
`sink.install(Level::INFO);` as a statement dropped the guard immediately and
captured nothing while the test still compiled. The attribute now sits on
`install`.

The comment above the `install` call in `harness.rs` restated the method's own
rustdoc and the `expect` string two lines below it.

AI Assistance: claude-opus-5 used for red-team review of the branch and these fixes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

observability: count every capability denial

1 participant