Skip to content

RFC: ABX skill description that loads on intent, not ABX vocabulary - #221

Draft
jordanlyall wants to merge 1 commit into
mainfrom
rfc/skill-description-intent
Draft

jordanlyall wants to merge 1 commit into
mainfrom
rfc/skill-description-intent

Conversation

@jordanlyall

Copy link
Copy Markdown
Member

Draft for discussion, not for merge yet.

Summary

I've been testing whether the abx skill loads when someone asks a coding agent for something ABX is good at, without saying ABX, NFT or blockchain. Think "a wedding guestbook that never disappears" or "certificates employers can verify even if the school shuts down."

Today's description mostly triggers on ABX vocabulary ("ABX collection", "mint", "NFT projects"). This draft rewrites only the description: field around user intent: permanence, ownership and verifiability. It keeps the capability list and adds a "not for ordinary apps" line so it doesn't fire on normal builds. The skill body is unchanged.

Curious whether this direction feels right, and what I dropped that should come back.

Results

Clean Claude Code (CLI 2.1.325, Opus 5.5). Each case runs from an empty folder with only the abx skill installed: no user CLAUDE.md, no other skills or plugins, no MCP servers, no memory. Graded on whether the Skill tool is invoked for abx. Both descriptions ran paired in the same run, 3 reps per case.

Held-out set, 25 cases written by a separate agent that never saw the rewrite:

Current This draft
Should fire (12 cases, 36 runs) 19/36 (53%) 35/36 (97%)
Should not fire (13 cases, 11 near-misses) 0 false fires 0 false fires

The near-misses share words with the triggers but mean something ordinary: "keep backups forever", "audit log staff can't edit", "users own their profiles", "pdf receipts", "permanent 301 redirect", "art portfolio gallery". Neither description fired on any of them.

The biggest gains were requests about lasting or proving with no collectible in the wording: a rec league trophy card kept forever (0/3 to 3/3), publicly timestamping a raw dataset (0/3 to 3/3), and an investigative piece that can't be quietly edited (0/3 to 3/3).

Literal requests still work. 8 prompts that name the tech ("mint an NFT collection", "deploy an ERC-721 contract", "nft on base at 0.01 eth", "make my song an NFT") plus 4 crypto requests that aren't ABX (Uniswap swap, wallet balance, trading bot, ERC-20 token):

Current This draft
Literal, should fire (24 runs) 20/24 24/24
Crypto but not ABX (12 runs) 0 false fires 2 false fires, both "create an ERC-20 token"

I also tried adding an explicit "NFTs, tokens, ERC-721 / ERC-1155" clause. It gained nothing on the literal set and fired on the ERC-20 prompt 3/3, so it's not in this draft.

A first set of 15 cases (which I wrote, and which the rewrite was shaped against) went 70% to 100%, also with 0 false fires. I'd weight the held-out numbers.

Open questions

  • Dropped from the current text to make room: "on supported networks", "shared resources", "managed or self-hosted" resolvers, "serve", "deploy", and "develop project ideas or ecosystem integrations". The last one may matter for builder and partner requests, which neither set tested. It fits back in under the 1,024-character limit.
  • ERC-20 requests load the skill 2/3 times. Possibly fine (the skill can say ABX doesn't do fungible tokens), or worth a "not fungible tokens" in the exclusion line. Untested.
  • "Put things on chain" as the opener vs. something that leads with "programmable digital objects", to match Position ABX as programmable digital objects, not only NFTs #217.

Verification

  • Paired eval as above. Happy to share the harness or rerun it on a different wording.
  • Not tested: Sonnet or Haiku, other agents (Codex, Cursor), or whether builds go better once the skill loads. This only measures loading.

Compatibility

  • Description text only. No CLI, contract or package API changes.
  • No Changeset yet, since this is for discussion. I'll add one if we go ahead.
  • Tests cover the changed behavior. (n/a, see eval above)
  • Public docs are updated, or this change does not affect them.
  • A Changeset is included, or no published package changes.
  • No credentials, local state, generated archives, or private planning/review material is included.

🤖 Generated with Claude Code

Rewrites only the description field around permanence, ownership and
verifiability, with an exclusion line for ordinary builds. Draft for
discussion; eval results in the PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
abx-docs Ready Ready Preview Nov 14, 2026 8:56pm UTC

Request Review

This branch was successfully deployed

1 active deployment
Preview — 0b6de811 Deployed Nov 14, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant