Declare and validate each skill's capabilities - #135
Merged
Merged
Conversation
Every skill's meta.yaml now carries a required, versioned `capabilities` block (schema 1): files read (none/diff/repo) and edited (none/repo), commands it may run, network use and why, credentials, agent tools, and files it may create. Anything undeclared is not requested; the declaration is for review, not enforcement. - skilldeck.capabilities validates the block (unknown or missing keys, other schema numbers, commands given as paths or with shell operators, artifact paths that are absolute, name a drive or home, use backslashes or have ./.. components). - The registry enforces the bundle rules: exactly regular meta.yaml and skill.md, rejecting symlinks (including a symlinked skill directory), subdirectories, undeclared executables (script suffix, execute bit, #!, binary headers) and any other file, plus skill.md links to files the skill cannot ship. provenance --verify and catalog apply the same rules. - Every adapter appends a "Declared capabilities" section to a skill that asks for more than reading files; the adapter contract skill declares its `git diff`, so the fixtures pin that section (contract sha256:d8b7d4463e84). - `skilldeck show <skill> --summary` and `skilldeck install --dry-run` preview a skill's source, build, digest check, deprecation and declared capabilities; the dry run also reports what installing would do and writes nothing. - `skilldeck catalog --json` reports `capabilities` (additive; schema_version stays 1). - All 13 bundled skills declare what their instructions ask for, with patch version bumps; plugin tree and content manifests regenerated. Closes #73 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HtiCGzpikMrkDYBkfQG5CX
- Render the "Declared capabilities" notice only for skills that ask for
more than a read-only review (an edit, a credential, an agent tool, an
artifact, or a command beyond read-only git); the ten read-only review
skills install byte-for-byte as before. Reword the notice for the agent
("Beyond reading the repository, this skill asks you to: ... It asks for
nothing else.") and rename Capabilities.beyond_reading to
beyond_review_baseline. The contract skill now declares
<the project's test command>, so the fixtures still pin the notice
(contract sha256:7c21e0856472).
- Loading ignores OS and editor leftovers (.DS_Store, Thumbs.db,
desktop.ini, __pycache__, ._*, .#*, *~, #*#, Vim swap files) so one stray
file no longer breaks every command; provenance --verify and catalog
still report them as unexpected files, with main's wording. .DS_Store is
gitignored.
- install --dry-run checks the nearest existing folder on the way to each
destination and reports the error a real install would hit.
- test-review runs a regression test against pre-change code only in a
temporary git worktree (declared); dependency-review runs pip-audit only
as `pip-audit --disable-pip` on fully pinned files, per pip-audit's
security model, and declares reading package registry pages.
- The link check matches src/href only inside HTML tags, skips indented
code blocks, and checks autolinks.
- The structure test knows common CLIs, so a new undeclared scanner, curl
or test run is caught; spans a skill only quotes are listed per skill.
- Declared commands reject interpreters given a script or inline code;
Windows junctions count as links; the catalog docs say a new capability
schema is a breaking catalog change; summary wording points to
provenance --verify and docs/verifying-releases.md.
Part of #73
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtiCGzpikMrkDYBkfQG5CX
Conflicts resolved keeping both sides: - CHANGELOG.md: #134's lifecycle Added bullets and this branch's capability and bundle-validation bullets; [Unreleased] ### Security still names no skill or adapter. - docs/authoring-skills.md: #134's Deprecating paragraph (CHANGELOG ### Deprecated, notice period, lifecycle.md) ends "Deprecating a skill", followed by this branch's Capabilities and "What a skill directory may hold" sections. - docs/catalog.md: #134's removed-skill sentence and this branch's capabilities bullet. - tests/test_catalog.py: uses #134's shared tests/_schema.py validator; the `enum` keyword this branch's catalog schema needs is ported there. tests/test_lifecycle.py's synthetic skills now carry the capability declaration that meta.yaml requires. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HtiCGzpikMrkDYBkfQG5CX
docs/lifecycle.md (#134) defines skill SemVer and what is breaking for skill authors; apply it to this branch: - dependency-review goes to 0.5.0, a minor bump: running pip-audit only on fully pinned files and adding registry-page lookups changes how it gathers evidence, which can change what it reports. test-review stays at 0.3.1, a patch: the temporary-worktree step only spells out a safe way to do what it already asked, and what it reports is unchanged. Both CHANGELOG entries say why. - Requiring `capabilities` and limiting a skill directory to meta.yaml and skill.md tightens rules existing metadata could fail, which the policy calls breaking for skill authors: add a **Breaking:** Changed entry. - docs/lifecycle.md's meta.yaml section mentions `capabilities`: its own schema number, and how a declaration change versions a skill. Part of #73 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HtiCGzpikMrkDYBkfQG5CX
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Skills didn't declare what they ask an agent to do, so a reviewer or installer had to read every instruction to find out. This adds a required, versioned
capabilitiesblock to every skill'smeta.yaml(schema 1). It covers:Anything not declared is not requested. The docs say plainly that this is a declaration for review, and that skilldeck can't enforce it inside an agent.
Validation (
skilldeck.capabilities,registry) rejects:~,\,.or...Bundle rules. A skill directory is exactly a regular
meta.yamlplus a regularskill.md. It rejects:#!or binary header;skill.mdto files the skill can't ship. This covers Markdown links, images, reference definitions, autolinks, and HTML tagsrc/href. Code, prose and indented blocks aren't checked.OS and editor leftovers (
.DS_Store, swap files, …) are ignored when loading, so one stray file doesn't break every command.provenance --verifyandcatalogstay strict and report them as unexpected files.tests/test_capabilities.pybuilds the adversarial bundles intmp_path.Preview.
skilldeck show <skill> --summaryandskilldeck install … --dry-runshow:provenance --verifyfor real verification;The dry run also shows what each install would do, or the error it would hit (including a destination folder that can't be created). It writes nothing.
Installed files. Adapters add a
## Declared capabilitiessection, addressed to the agent, only to skills that ask for more than a read-only review: an edit, a credential, an agent tool, an artifact, or a command beyond read-only git. The ten read-only review skills install byte for byte as before; onlydependency-review,loggingandtest-reviewgain the section. The adapter contract skill declares its project test command, so the fixtures pin the section (contractsha256:7c21e0856472).Catalog.
catalog --jsongains an additivecapabilitiesfield.schema_versionstays 1; a future capability schema would be a breaking catalog change.The 13 bundled skills each declare what their instructions ask for, with a version bump: patch for the declaration alone, and
dependency-reviewgoes to 0.5.0 (a minor bump) because its instructions changed.test-reviewnow runs a regression test against the pre-change code only in a temporarygit worktree.dependency-reviewrunspip-auditonly aspip-audit --disable-pipon fully pinned files, following pip-audit's security model, and declares reading package registry pages.Structure test. It keeps the declared commands and the skill bodies in sync, including commands from common CLIs that no skill declares yet. The plugin tree and manifests are regenerated.
Breaking for skill authors, per
docs/lifecycle.md:meta.yamlnow requirescapabilities, and a skill directory may hold onlymeta.yamlandskill.md.An independent review found two problems, both fixed here:
.DS_Storebroke every command.It also found:
--dry-runreporting "would install" when a destination folder couldn't be created;Not done here: the golden-diff evals weren't re-run, because they're paid. The
dependency-reviewandtest-reviewbodies changed, and three skills gained the section, so a maintainer should run them by hand.Closes #73
Tracking: #94
Type of change
Checklist
uv run --extra dev ruff check . && uv run --extra dev ruff format --check . && uv run --extra dev mypy && uv run --extra dev pytest(1137 passed, also with-W error::EncodingWarningand--resolution lowest-direct)CHANGELOG.mdentry under## [Unreleased]versioninmeta.yaml🤖 Generated with Claude Code
https://claude.ai/code/session_01HtiCGzpikMrkDYBkfQG5CX
Generated by Claude Code