Skip to content

Fix the failure modes found in first field use - #1

Merged
gurnben merged 1 commit into
mainfrom
fix/first-field-use-failures
Oct 20, 2026
Merged

gurnben merged 1 commit into
mainfrom
fix/first-field-use-failures

Conversation

@gurnben

@gurnben gurnben commented Oct 20, 2026

Copy link
Copy Markdown
Member

Summary

Three runs against github.com/gurnben/crush (two secure-code-audit, one threat-model) exposed a cluster of issues, all fixed here:

  • 401 on every audit run: the registry floor model resolved to claude-mythos-5, which crush routed to an Anthropic provider with an invalid key. Runs now fail fast when the harness can verify the model is not runnable (crush: checked against crush models), harness failures surface the stderr tail so the provider error is visible without opening the journal, and both are recorded in result.json as harness_error.
  • Healthy threat-model run aborted at minute 9: crush emits output in rare large chunks, so the monitor looked frozen and a single ctrl+c killed the run. The status bar now shows time since last output, and aborting requires a confirming second ctrl+c within six seconds. Force-quitting the monitor tears the harness down and joins the runner first, closing the "run produced no result" race and the send-on-closed-channel risk.
  • Output didn't register: aborted and failed runs now still discover and validate their artifacts (partial output and checkpoint state matter most exactly when a run dies), and the run summary prints after the interactive monitor, not only in plain mode.
  • Budget policy parse error on every run: the parser now accepts the shipped sequence-form bands: (- {at: 1.5, action: warn}) that the real deployment file uses.

Verification

  • 34 tests (race detector on), go vet and staticcheck clean; new tests cover the model-check fail-fast, stderr-tail surfacing, partial-output discovery, sequence-form bands, and crush model matching.
  • Full-fidelity dry run against a temp workspace wired to the real venv, registry and budget policy with a fake harness: floor resolution, budget caps (2.5M/1M), artifact discovery and real schema validation all behave.
  • Reproduced the original 401 live before fixing; the failure path now prints the provider error and guidance inline.

Note: the companion deployment fix (mapping the deep-audit and threat-model roles to rits/zai-org/glm-5-3 in the local model-registry.yaml, the mythos-class model this crush can authenticate to) was applied to ~/.traust/config directly — the registry is designed for exactly that deployment customization.

💘 Generated with Crush

Three runs against github.com/gurnben/crush exposed a cluster of issues:

The audit runs 401'd immediately: the registry floor model resolved to
claude-mythos-5, which crush routed to an Anthropic provider with a bad
key. Runs now fail fast when the harness can verify the model is not
runnable (crush checks `crush models`), harness failures surface the
stderr tail so the provider error is visible without opening the journal,
and both are recorded in result.json as harness_error.

The threat-model run was healthy but aborted: crush emits output in rare
large chunks, so the monitor looked frozen for minutes and a single
ctrl+c killed a 9-minute audit. The status bar now shows time since last
output, and aborting requires a confirming second ctrl+c within six
seconds. Force-quitting the monitor tears the harness down and joins the
runner first, closing the "run produced no result" race.

Aborted and failed runs now still discover and validate their artifacts
(partial output and checkpoint state matter most exactly when a run dies),
and the run summary prints after the interactive monitor, not only in
plain mode.

Also: budget-policy parsing accepts the shipped sequence-form bands
(- {at, action}) that the real deployment file uses.

Verified end to end against a temp workspace wired to the real venv,
registry and budget policy with a fake harness; 34 tests, vet and
staticcheck clean.

💘 Generated with Crush

Assisted-by: Crush:rits/zai-org/glm-5-3
@gurnben
gurnben requested a review from a team as a code owner October 20, 2026 18:43
@gurnben
gurnben merged commit 8ce9f32 into main Oct 20, 2026
2 checks passed
@gurnben
gurnben deleted the fix/first-field-use-failures branch October 20, 2026 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant