Skip to content

fix: three papercuts the end-to-end run turned up - #204

Merged
vicenteliu merged 1 commit into
mainfrom
fix/papercuts-from-the-e2e-run
Aug 19, 2026
Merged

fix: three papercuts the end-to-end run turned up#204
vicenteliu merged 1 commit into
mainfrom
fix/papercuts-from-the-e2e-run

Conversation

@vicenteliu

Copy link
Copy Markdown
Owner

Three smaller defects from the same end-to-end run. Unrelated to each other, all small, all real.

1. Editing the model list from the admin UI deleted the comments inside it

Found by using the product as intended: adding two models through PUT /api/admin/playbook-models stripped three comments from the checked-in playbook, including

    params:
      # Sonnet 5 / Opus 5 reject temperature / top_p / top_k (HTTP 400).
      max_tokens: 8192

which exists precisely to stop someone re-adding a field that 400s.

write_playbook_models splices the file around the model: and extra_models: blocks, so comments outside survived — but both blocks are re-rendered from parsed data, so comments inside could not. The function docstring claimed flatly "All other content — comments, key order, formatting — is preserved."

Two ruamel details made this harder than it sounds, and I got the first fix wrong before catching it:

  • A comment before the first key of a nested mapping attaches to the parent's key. The one above max_tokens: is stored on params, not on max_tokens.
  • The heading of the next top-level section is parked on the last key of this block. Carrying it forward printed # Loop bounds. twice — once where it belongs and once inside the rewritten block. Column separates them cleanly: an inner comment is indented (col=6), a section heading sits at col=0.

An unchanged value now also keeps the scalar ruamel parsed, so an edit no longer restyles effort: "high" into effort: high. An unchanged rewrite is byte-identical, which is now a test.

2. --model on workingset distil could not switch providers

model_name = model or target.name
provider = make_provider(target.provider_id, kind=target.kind)   # ← unaffected by --model

It replaced only the model name and kept the tier's provider. So passing the id that GET /api/models returns and the UI uses:

$ opspilot workingset distil ws_… --model anthropic/claude-sonnet-5
ProviderError: Anthropic API error: 404 - {'type': 'not_found_error',
  'message': 'model: anthropic/claude-sonnet-5'}

— as a full Python traceback, because only NotDistillableError was caught. And naming another provider's model would have sent it to the wrong API entirely.

It now resolves against the playbook's model list, accepting either provider/name or the bare name, refuses an unknown one with the list of valid ids, and exits cleanly on ProviderError. Both verified against the live instance.

3. The API dropped working_set_id from every Consultation

_con() serialized id / author / title / created_at / updated_at / pinned_reason / session_id — but not working_set_id.

ADR-0036 makes the Working set the chain of Consultations on one problem, and that chain is what distillation reads, so this was the domain's central relationship missing from its own API. Nothing was broken at the time — distillation calls consultations.for_working_set() on the store directly and the web client never declared the field — which is exactly why it survived.

Verification

6 new tests. The playbook ones use a fixture with comments inside the model blocks; the existing fixture put every comment outside them, which is why it stayed green while the real file lost comments on every edit.

pytest -m "not slow and not requires_ollama and not requires_api_key" — 1317 passed. No protected path touched.

🤖 Generated with Claude Code

**Editing the model list from the admin UI deleted the comments inside it.**
Adding two models through `PUT /api/admin/playbook-models` stripped three
comments from the checked-in playbook, including

    # Sonnet 5 / Opus 5 reject temperature / top_p / top_k (HTTP 400).
    # Opus 5 rejects a token budget ... so depth is `effort`.

which exist to stop someone re-adding a field that 400s. `write_playbook_models`
splices the file around the two model blocks, so comments *outside* survived,
but both blocks were re-rendered from parsed data, so comments *inside* could
not. They are now carried across for every key that survives the rewrite.

Two ruamel details made that harder than it sounds. A comment before the *first*
key of a nested mapping is attached to the parent's key, not to the key it
visually precedes — so the one above `max_tokens:` lives on `params`. And the
heading of the *next* top-level section is parked on the last key of this block;
carrying it forward printed `# Loop bounds.` twice, once where it belongs and
once inside the rewritten block. Column separates them: an inner comment is
indented, a section heading sits at column 0. An unchanged value now also keeps
the scalar ruamel parsed, so editing the list no longer restyles `effort: "high"`
into `effort: high`, and an unchanged rewrite is byte-identical.

**`--model` on `workingset distil` could not switch providers.** It replaced
only the model *name* and kept the tier's provider, so passing the id the API
and UI use — `anthropic/claude-sonnet-5` — sent that whole string to Anthropic
and came back `404 not_found_error`, as a full traceback because only
`NotDistillableError` was caught. It now resolves against the playbook's model
list, accepting either form, refuses an unknown one with the list of valid ids,
and `ProviderError` exits cleanly.

**The API dropped `working_set_id` from every Consultation it returned.** ADR-0036
makes the Working set the chain of Consultations on one problem, and that chain
is what distillation reads, so this was the domain's central relationship
missing from its own API. Nothing was broken at the time — distillation reads
the store directly and the web client never asked for the field — which is
exactly why it survived.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vicenteliu
vicenteliu merged commit dcfd1ae into main Aug 19, 2026
4 checks passed
@vicenteliu
vicenteliu deleted the fix/papercuts-from-the-e2e-run branch August 19, 2026 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant