Skip to content

fix(vendor-neutrality-score): never count partial-coverage tools as backends - #1381

Merged
potiuk merged 1 commit into
apache:mainfrom
potiuk:fix/vendor-neutrality-partial-coverage
Sep 25, 2026
Merged

potiuk merged 1 commit into
apache:mainfrom
potiuk:fix/vendor-neutrality-partial-coverage

Conversation

@potiuk

@potiuk potiuk commented Sep 25, 2026

Copy link
Copy Markdown
Member

Summary

  • tools/vendor-neutrality-score counted every **Kind:** implementation tool as a backend vendor and never read the **Coverage:** qualifier. docs/labels-and-capabilities.md says a partial tool "must not be counted as a complete/selectable backend", but nothing enforced it. A read-only foundation for a new forge (for example the GitLab bridge in feat(tools/gitlab): add read-only GitLab bridge foundation (Part of #305) #1369) would raise the contract counts in docs/vendor-neutrality.md as if it were a full backend, even after its README declared **Coverage:** partial.
  • The scorer now reads **Coverage:** (partial or partial-read-only, with or without backticks), keeps partial tools out of the vendor count, lists them in the generated Basis column (partial foundation, not counted: …) and in --json (partial_implementations), and fails on unknown values, just as it already does for **Kind:**.
  • No score changes today. Atlassian stays counted for contract:tracker / contract:change-request through jira / jira-patch, and bitbucket now appears as not counted.
  • The coverage-qualifiers section now also documents partial, which tools/bitbucket/README.md already uses, and states that the scorer enforces it.

Type of change

  • Python package (tools/*/ with pyproject.toml)
  • Documentation (docs/)

Test plan

  • uv run --directory tools/vendor-neutrality-score pytest: 36 passed (3 new: partial not counted; README parsing incl. backticked value; unknown value raises)
  • ruff check, ruff format --check, mypy for the tool: clean
  • vendor-neutrality-score --in-place regenerated the doc block (Basis column only; counts unchanged)
  • prek run --all-files: all hooks pass except workspace-pytest for dev, agent-isolation and pr-management-stats, which fail identically on unmodified main in my sandboxed environment (socket / subprocess denial, local plugin tree). CI runs them unsandboxed.

Linked issues

Follow-up from the review of #1369.

Generative AI disclosure

This change was drafted with Claude Code (Claude Opus 5) and reviewed by me before submission.

🤖 Generated with Claude Code

…ackends

The scorer counted every `**Kind:** implementation` tool as a backend
vendor and ignored the `**Coverage:**` qualifier, although
docs/labels-and-capabilities.md says a partial tool "must not be
counted as a complete/selectable backend". A read-only foundation for a
new forge would therefore raise the contract counts in
docs/vendor-neutrality.md as if it were a full backend.

The scorer now reads `**Coverage:**` (`partial` or `partial-read-only`,
backticks allowed), keeps partial tools out of the vendor count, lists
them in the Basis column and the JSON output, and rejects unknown
values. No score changes today: Atlassian stays counted through
jira / jira-patch, and bitbucket is now shown as not counted.

The coverage-qualifiers section also documents `partial`, which the
bitbucket README already uses, and the scorer's behaviour.

Generated-by: Claude Opus 5
@potiuk
potiuk merged commit c824646 into apache:main Sep 25, 2026
9 checks passed
@potiuk potiuk added substrate:analytics Tool substrate: read-only metrics / dashboards / renderers substrate:framework-dev Tool substrate: build / validate / eval the framework itself family:tools tools/* labels Sep 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

family:tools tools/* substrate:analytics Tool substrate: read-only metrics / dashboards / renderers substrate:framework-dev Tool substrate: build / validate / eval the framework itself

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant