Publish an agent compatibility matrix backed by adapter contract tests - #132
Merged
Merged
Conversation
Add docs/compatibility.md, linked from the README and docs/adapters.md. It covers every adapter: the five native ones and the copilot-prompt, cursor-rule and kiro-steering legacy adapters. Each row gives a status (tested, supported or experimental, defined on the page), the project and global paths, the environment variables that move them, how the skill is invoked, the minimum agent version, and the date, agent version and vendor sources it was checked against. The research behind it was done on 2026-09-23, and whatever it couldn't confirm is marked unverified. The page also sets out how skilldeck responds when a vendor deprecates or moves a skill location or format. tests/fixtures/adapter-contracts/ pins each adapter's exact rendered file (stamp included) for one synthetic skill, its project and global paths, its --scope global error if it is project-only, and its behaviour with each config-directory variable set to an absolute path, left empty or set to a relative path. tests/test_adapter_contracts.py installs the skill with every adapter into temporary directories and compares the results byte for byte. The fixtures are marked -text in .gitattributes so Windows checkouts keep them exact. The matrix embeds a sha256 digest of the fixtures, and the changelog must mention its first 12 characters, so a format change fails CI until the matrix and changelog are updated. The error for an unsupported scope now names what works instead. For example, "Use --scope project, or --agent cursor (Agent Skills), which supports --scope global". Closes #79. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HtiCGzpikMrkDYBkfQG5CX
- Only install's unsupported-scope error suggests the native adapter
(--agent cursor). For status, uninstall and update, that adapter would
act on different files, so those commands name only --scope project.
Adapter.install checks the scope first, so direct API use gets the same
message.
- The contract fixture directory must hold exactly the contract's files.
The digest covers only those files, and covers contracts.json as
canonical JSON without its _about notes.
- The matrix test now checks each row cell by cell:
- project and global paths, with "not supported" and "n/a" for
project-only adapters
- "Moved by" names exactly the variables that move the global install,
each with its target, and mentions the ones that don't
- both quoted scope errors match the real messages
The docs and CLAUDE.md say what is enforced and what is kept by hand.
- The CHANGELOG digest mention must be in [Unreleased] or the newest dated
section, so the check still holds right after prepare_release cuts a
release. Tested with synthetic changelogs and with the real
cut_changelog.
- Cursor and cursor-rule are experimental: their only evidence is
@cursor/sdk. Experimental is now defined to cover SDK-only evidence.
Claims about the Cursor app and CLI, including env overrides, are marked
unverified, and cursor-rule's invocation gets the same caveat.
- Codex's $name resolution is hedged: its two selection paths differ, and
which surface uses which is unverified. docs/adapters.md is hedged too.
Part of #79.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtiCGzpikMrkDYBkfQG5CX
# Conflicts: # CHANGELOG.md # CLAUDE.md # src/skilldeck/cli.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Adds
docs/compatibility.md, a public compatibility matrix linked from the README anddocs/adapters.md. It covers all eight adapters: the five native ones plus thecopilot-prompt,cursor-ruleandkiro-steeringlegacy adapters. For each adapter it gives:Anything a primary source doesn't confirm is marked unverified. The page also explains how the project responds when a vendor deprecates or moves a location or format, including urgent releases.
CI now pins each adapter's contract.
tests/fixtures/adapter-contracts/holds a synthetic skill, acontracts.json, and the exact rendered file every adapter should produce, stamp included.contracts.jsonrecords paths, scope-error text (forinstalland for other commands), the expected frontmatter, and the behaviour with each config-directory variable set to an absolute path, empty, or relative.tests/test_adapter_contracts.py:-text, no symlinks;<!-- adapter-contract: sha256:… -->line to match a digest of the contract files.contracts.jsonis hashed as canonical JSON without its_aboutnotes;[Unreleased]or, once a release is cut, in the newest dated section. This is tested againstprepare_release.cut_changelog.Unsupported scopes now fail with an actionable message.
install --agent cursor-rule --scope globalsays "Use --scope project, or --agent cursor (Agent Skills), which supports --scope global".status,uninstallandupdatename only--scope project, because the native adapter would act on different files.Review. An independent review raised eight findings, all fixed here:
cursor-ruleinvocation caveat is added;$nameresolution is hedged as unverified;_aboutis excluded from the digest.Closes #79
Tracking: #94
Type of change
Checklist
uv run --extra dev ruff check . && uv run --extra dev ruff format --check . && uv run --extra dev mypy && uv run --extra dev pytest(817 passed, also with-W error::EncodingWarningand with--resolution lowest-direct)docs/adapters.md, newdocs/compatibility.md, CLAUDE.md)CHANGELOG.mdentry under## [Unreleased]versioninmeta.yaml(no skill content changed)🤖 Generated with Claude Code
https://claude.ai/code/session_01HtiCGzpikMrkDYBkfQG5CX
Generated by Claude Code