Skip to content

feat: add native Anthropic support to ToolHive gateway - #1339

Open
samuv wants to merge 1 commit into
mainfrom
feat/toolhive-anthropic
Open

feat: add native Anthropic support to ToolHive gateway#1339
samuv wants to merge 1 commit into
mainfrom
feat/toolhive-anthropic

Conversation

@samuv

@samuv samuv commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • register toolhive-anthropic as a distinct native Anthropic Messages provider while preserving toolhive as the OpenAI Responses default
  • derive proxy/direct Anthropic paths safely, share the direct-mode OIDC token source, and strip conflicting authentication headers
  • keep protocol catalogs, status, metadata, and last-known-good state independent, with deterministic model inventory and ToolHive-family UI handling

Protocols

  • toolhive: GET /v1/models, POST /v1/responses
  • toolhive-anthropic: GET /anthropic/v1/models, POST /anthropic/v1/messages

Verification

  • task fmt
  • task docs
  • task site:build
  • task api:check
  • task lint
  • task test
  • go run ./cmd/mecademo
  • focused internal/app and cmd/mecatui/ui tests

task ac-trace-strict currently stops on the pre-existing docs/acceptance/callmcpwithquery-broker-support.md status proposed, which the checker does not recognize.

@samuv
samuv force-pushed the feat/toolhive-anthropic branch from 8236635 to bb9b5a5 Compare September 10, 2026 12:20
@samuv samuv changed the title Add native Anthropic support to ToolHive gateway feat: add native Anthropic support to ToolHive gateway Sep 10, 2026
@kantord

kantord commented Sep 10, 2026

Copy link
Copy Markdown
Member

Cursory review (Standards + Spec)

Fully or partially written by an AI agent.

Two-axis review of main...HEAD: does the diff follow this repo's documented standards, and does it match the acceptance plan / ADR 0326?

Standards

Hard violations

  1. docs/design/PRODUCTION-READINESS.md — the sole shipped/deferred tracker per AGENTS.md's doc-lifecycle header — isn't updated even though a new provider identity (toolhive-anthropic) ships here. docs/acceptance/toolhive-native-anthropic.md waives the process spine, but that doesn't cover the tracker update itself.

Judgement-call smells

  • Feature Envycmd/mecatui/ui/models_surface.go:253-255 infers routing mode via strings.Contains(hint, "--toolhive-llm-mode proxy"), reaching into a free-text diagnostic string owned by internal/app/registry.go. A copy edit to the hint silently breaks this with no compiler signal.
  • Duplicated/drifting hint logicstatusHintFor (mode-unaware) vs. the new toolhiveStatusHint (mode-aware) in registry.go. modellister.go:637-640 computes one then overrides with the other; recordSuccess:722 still only calls the old one. Landmine if the two hint maps ever diverge.
  • Duplicated predicate across packagesisToolhiveProviderID (TUI) and isToolhiveProvider (internal/app) reimplement the same "is this ToolHive" check with independent string literals; a third protocol member needs both updated in lockstep.
  • Speculative Generality (pre-existing)toolhiveIntent.oidcConfigured is set but never read by production code, only referenced in test literals.

Spec

(a) Missing/partial: None found. All three acceptance scenarios (registration, discovery-path derivation, metadata threading, shared token source, transport stripping, independent outcomes, default compatibility, status/UI presentation) have direct test coverage, including the requirement to carry output/thinking limits into the emitted Anthropic max_tokens/thinking mode.

(b) Scope creep — The spec asks for concurrent fetch only across the ToolHive pair ("Fetch both ToolHive protocol entries concurrently under the existing operation-wide bounds"). probeToolhive correctly scopes to that pair, but modellister.go's liveModelSnapshot/refreshStaleModels were rewritten to launch a goroutine per every provider in reg.Available() — openai, anthropic, openai-codex, custom providers included. Neither the ADR nor the acceptance plan asks for this, and only the ToolHive case (TestToolhiveProtocolRefreshesStartConcurrently) is tested.

(c) Implemented-but-questionable: Same fragile substring-match flagged under Standards — providerStatusLine derives proxy-vs-direct UI state from matching "--toolhive-llm-mode proxy" inside the hint string rather than a structural field (e.g. a toolhiveMode on client.ProviderStatus). No test ties the two together.


Summary: Standards — 1 hard violation (missing tracker update) + 4 judgement-call smells, worst being the hint-string coupling between registry.go and models_surface.go. Spec — 0 missing requirements, 1 scope-creep item (concurrency widened to all providers, untested beyond ToolHive), 1 questionable implementation (same fragile substring-match issue), worst being the untested concurrency generalization to non-ToolHive providers.

Worth a decision: was widening background/on-demand refresh concurrency to all providers (not just ToolHive) intentional? If so, it probably deserves its own test coverage and a line in the ADR/acceptance doc; if not, scope it back to the ToolHive pair.

@kantord kantord left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, see the comments above

@samuv
samuv force-pushed the feat/toolhive-anthropic branch from bb9b5a5 to 2113e55 Compare September 10, 2026 14:14
@samuv

samuv commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the review findings in the rebased branch:

  • added ADR 0326's shipped state to PRODUCTION-READINESS.md;
  • removed TUI inference from remediation text and made the unreachable label routing-neutral;
  • consolidated proxy/direct remediation through one entry-aware status helper;
  • scoped concurrent catalog fetches to the two ToolHive protocol providers, with regression coverage proving unrelated providers remain sequential;
  • added stale-refresh concurrency coverage and removed the unused intent field.

The app and TUI keep package-local ToolHive-family predicates because the TUI cannot import internal/app; both IDs remain directly covered by tests.

Verified with task test:golden, task docs, task lint, task test, focused package tests, and go run ./cmd/mecademo.

@samuv
samuv force-pushed the feat/toolhive-anthropic branch from 2113e55 to 7674de7 Compare September 10, 2026 14:45
@samuv

samuv commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current origin/main (1264f5ad2) and aligned the PR with ADR 0321's documentation ownership:

  • docs/usage.md and docs/usage/ remain unchanged compatibility pointers;
  • shared provider-selection behavior now lives in user-docs/features/choose-models.md;
  • proxy/direct routing, OIDC, TLS, and daemon flags remain in user-docs/building/deployment/mecated.md.

Verified with task docs, task site:build, task lint, task test, and the offline demo.

Co-Authored-By: OpenAI Codex <codex@openai.com>

Signed-off-by: Samuele Verzi <samu@stacklok.com>
@samuv
samuv force-pushed the feat/toolhive-anthropic branch from 7674de7 to ff721eb Compare September 11, 2026 08:29
@samuv samuv self-assigned this Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants