Add claude-api Files upload regression example - #14
Open
GBX-Max1220 wants to merge 3 commits into
Open
GBX-Max1220 wants to merge 3 commits into
GBX-Max1220 wants to merge 3 commits into
Conversation
Adds examples/claude-api-files-purpose, pinning anthropics/skills at 5128e1865d670f5d6c9cef000e6dfc4e951fb5b9 (the commit before the upstream Files API repair) and generating a deterministic static contract test that detects the unsupported `purpose` parameter in Files upload examples. Upstream repair recorded: d230a6dd6eb1a0dbee9fec55e2f00a96e28dff81. Baseline contract test fails on four examples (curl/managed-agents.md, shared/managed-agents-client-patterns.md, shared/managed-agents-environments.md, typescript/managed-agents/README.md); removing the four `purpose` arguments makes it pass. The contract test is structural: it does not call Anthropic's API, does not run upstream code, and does not assert server error wording or live behavior. Scope: examples/ only. No SkillHone core changes.
…s case The case could be reproduced but not verified: confirming that the contract test is satisfiable by the real fix meant hand-editing the four reported examples. prepare_fixture.py now emits repair.patch into the fixture at preparation time, taken from upstream repair d230a6dd6eb1a0dbee9fec55e2f00a96e28dff81. Before writing it, the script fetches that commit at depth 2 and checks that its parent is the pinned defect commit 5128e1865d670f5d6c9cef000e6dfc4e951fb5b9, that its skills/claude-api tree is 284049f382b3c2b1989353637e0b3848dad0f09f, and that it changes exactly the four Skill files the test reports. After committing the fixture it checks the patch still applies. The diff is produced with --relative=skills/claude-api so "git apply repair.patch" works from the fixture root. Verified: the baseline test fails on the same four examples and passes after the patch, with the four patched files byte-identical to the upstream repair blobs and the remaining Skill files unchanged. The patch is generated into the fixture only; no upstream content is added to this repository. README and the generated PROVENANCE.md both record that it is a verification aid an evaluation run should withhold from the system under test. Scope: examples/claude-api-files-purpose/ only. No SkillHone core changes.
prepare_fixture.py wrote the materialized Skill tree straight into the user-supplied destination, so a check that failed afterwards stranded a half-built fixture there, and the next run then refused to touch it. Everything now happens in a staging directory beside the destination: fetch, tree, parentage and scope verification, patch generation, the fixture's own commit, and the git apply --check. Only once all of it passes does a same-filesystem rename publish the fixture, and the finally block removes the staging tree on any failure, including an interrupt. Consequence: the destination's parent must already exist, since nothing is created before the build has succeeded. Verified against the real upstream commit: the happy path still fails the baseline on the same four examples and passes after git apply repair.patch; injected download, tree, parentage, scope and apply-check failures each leave no destination and no staging behind; SIGINT mid-fetch leaves nothing and the same path reruns cleanly; a destination placed on a different filesystem than TMPDIR still publishes by rename. Scope: examples/claude-api-files-purpose/prepare_fixture.py only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a reproducible SkillHone case for stale Files API upload guidance in Anthropic's
claude-apiSkill.What changed
5128e1865d670f5d6c9cef000e6dfc4e951fb5b9.purposeparameters in upload examples.d230a6dd6eb1a0dbee9fec55e2f00a96e28dff81.Reproduction
The baseline test fails with exit 1 and reports four unsupported
purposeparameters:curl/managed-agents.mdcode block 15: multipart purpose fieldshared/managed-agents-client-patterns.mdcode block 8: SDK purpose argumentshared/managed-agents-environments.mdcode block 3: SDK purpose argumenttypescript/managed-agents/README.mdcode block 11: SDK purpose argumentAfter applying the generated repair.patch to the fixture, the regression test passes (exit 0).
Validation and results
pnpm test: 31 passed, 0 failed.python3 skills/skillhone/scripts/quality/static_check.py skills/skillhone: passed.python3 skills/skillhone/scripts/quality/static_check.py skills/skillhone-auto-optimization: passed.gitleaks dir . --no-banner --redact: no leaks found.gitleaks git . --no-banner --redact: no leaks found (22 commits).git diff --check: passed.Effect and limits
Changes are limited to
examples/claude-api-files-purpose/. No SkillHone core files changed. The test is a static documentation contract; it does not call Anthropic's API or verify live server behavior.Review checklist
git diff --check.