Repository navigation
feat(ai-provider)!: provider catalog, protocol registry, plugin text protocols and OpenAI Decisions - #176
Merged
Conversation
Adapters returned each provider's own word (`end_turn`, `max_tokens`, `tool_use`) and three consumers translated it separately. On a call that is not streamed the runtime read Anthropic's `max_tokens` as a normal end, so a cut answer was kept as complete. The gateway now returns `stop`, `length`, `tool_calls`, `content_filter`, `error` or `other` and keeps the provider's word in `rawFinishReason`, the structure the AI SDK uses (`unified` / `raw`). One table in `@covel/shared` (`unifyFinishReason`) serves the gateway, the runtime and world generation. BREAKING CHANGE: `ctx.gateway` results carry the unified finish reason. A plugin that compared it with a provider's word must compare with the unified value.
…failures Transport retries: 408 and 409 are sent again like 429 and 5xx, `x-should-retry` decides when a provider sets it, and `retry-after-ms` is read before `Retry-After`. A wait of more than 60 seconds, or more than the call has left, is not waited for: the answer returns as rate limited and the gateway can use a backup model. Before, the call waited until its budget ran out and ended with an error that allows no backup. A connection test sends one request (`transportRetry: false`), tests an embedding model with a vector call, and reports `errorKind` beside the provider's words. `classifyProviderFailure` reads the kind from the status, the provider's error code and the cause chain of a failed connection, so a refused connection names its host and port instead of `fetch failed`.
…services, model lists Protocol IDs were copied by hand in seven places (the plugin SDK's copy had lost Gemini) and each protocol's rules were `if (protocol === ...)` branches in three files. The IDs, their names and their output kind now come from one table in `@covel/shared`, and a `ProtocolDefinition` holds a protocol's reasoning fields, provider options, optional parameters, media-wire rule and model list. The built-in providers grow from 8 to 22: nine more cloud providers and five services on this machine (Ollama, LM Studio, llama.cpp, vLLM, LiteLLM). The add-provider dialog picks one by name, an `llm.toml` slot of a built-in provider needs only `provider` and `model`, and a model on a loopback address is ready without an API key. `POST /api/ai/models` reads an endpoint's model IDs for the settings page.
`covel.registerWires({ text: [{ id, generateText, streamText }] })` adds a
request and response format beside the built-in protocols. A model whose
`protocol` is `<pluginId>/<wireId>` uses it for every text call; `llm.toml`,
the request validators and the settings page accept the ID, and
`GET /api/ai/protocols` lists the registered ones. Object calls go through
`generateText` with the JSON Schema as an instruction.
The plugin SDK gains the wire's types and the `providerOptions` field of
`ctx.gateway.generateText`, which the runtime already passed on.
`openai-decisions-v1` answers `gateway.evaluate()` through OpenAI's `POST /v1/decisions` (public beta, model `gpt-6-luna`). The public question and answer types stay as they are; the adapter converts at the wire: - `state` goes as `input` text, a JSON value serialized - the question map becomes a list with each name, and the answer list is keyed back by name - a boolean is a `predicate` whose true/false rubrics join `instructions` - choice options and score levels are lists; a choice needs 2-255 options - probability lists become the maps the other protocols return, under the same checks A question the model declines fails the call as a refusal, since the contract has an answer for every question. Usage carries cache reads and writes, and per-question confidence goes to `providerMetadata.openai`. The `openai` provider suggests this protocol for an evaluation model. Field names follow the API reference; no call was made with a real key.
ackness
added
the
breaking-change
9 of 19 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary / 摘要
Adding a provider or a protocol meant editing a protocol list copied by hand in seven places, and each protocol's rules were
if (protocol === ...)branches in three files. A player who wanted a provider outside the eight built-in ones had to look up its endpoint and pick a protocol, and a local service without an API key was shown as not configured. This PR reworks the provider module so both are one entry, and brings its formats in line with the AI SDK and the official provider SDKs.Five commits, each of which passes the gates on its own:
feat(ai-provider)!one finish-reason vocabulary. Adapters returned each provider's own word and three consumers translated it separately. On a call that is not streamed, Anthropic'smax_tokenswas read as a normal end, so a cut answer was kept as complete. The gateway now returns the unified reason and keeps the provider's word inrawFinishReason(the AI SDK'sunified/raw).fix(ai-provider)retry rules and failure kinds. 408 and 409 are retried,x-should-retryandretry-after-msare read, and a wait longer than 60 seconds or than the call has left returns at once so a backup model can be used. A connection test sends one request, tests an embedding model with a vector call, and reportserrorKind(unreachable,auth,quota,rate_limited, …) so the settings page can say what to do.feat(ai-provider)protocol table, provider catalog, local services, model lists. Protocol IDs, names and output kind come from one table in@covel/shared; aProtocolDefinitionholds a protocol's reasoning fields, provider options, optional parameters and model list. Built-in providers grow from 8 to 22 (nine cloud providers; Ollama, LM Studio, llama.cpp, vLLM, LiteLLM), the add-provider dialog picks one by name, anllm.tomlslot of a built-in provider needs onlyproviderandmodel, a model on a loopback address is ready without a key, andPOST /api/ai/modelsreads an endpoint's model IDs.feat(plugins)a plugin can register a text protocol.covel.registerWires({ text: [{ id, generateText, streamText }] }); a model whoseprotocolis<pluginId>/<wireId>uses it.GET /api/ai/protocolslists them for the settings page.feat(ai-provider)OpenAI Decisions.openai-decisions-v1answersgateway.evaluate()throughPOST /v1/decisions(public beta,gpt-6-luna). The public question and answer types do not change.Type of change / 变更类型
Breaking changes / 破坏性变更
finishReasonofctx.gateway.generateText/generateObjectand of the gateway's results is the unified value; the provider's word moved torawFinishReasonon gateway results. The bundledhistory-compactionplugin was the only one that compared a provider's word and is updated.Retry-Afteris no longer waited for. A call whose provider asks for more than 60 seconds, or more than the call's remaining budget, ends as rate limited (and may use a backup model) instead of waiting until the budget runs out.GROQ_API_KEY) attaches to that provider's official origin, under the same rule as the existing built-in providers.No session, snapshot, queued job or world has to be recreated.
Verification / 验证方式
pnpm check— on each of the five commits, in a clean worktreepnpm test— on each of the five commits, in a clean worktreepnpm test:pg— no store changepnpm e2e:smoke/pnpm e2e— not runpnpm validate:plugin/pnpm validate:world— covered bypnpm check; no manifest changepnpm e2e:verify— not runclassifyProviderFailureagainst real endpoints (a closed port, a host that does not exist, a wrong key) returnedunreachablewith the host and port,unreachable, andauth.Not verified / 未验证:
pnpm e2e:replaycould not run because the repository has no recording forlantern-barrow.Related issue / context / 关联
@ai-sdk/provider4.0.26 andai7.0.136 sources for finish reasons, retries and the decision model; OpenAI Decisions guide and API reference.baseUrlis accepted on every deployment tier, so ondemo/commerciala request-scoped preset can point the server at its own localhost. Telling an operator's loopback endpoint from a request's needs its own change.Docs sync / 文档同步
docs/reference/updated:slots.md(protocols, providers, adding one, finish reasons, retries, failure kinds, connection test),api.md(/api/ai/models,/api/ai/protocols, pingerrorKind),plugin-extensions.md(text wires),evaluation.md(OpenAI Decisions),media-store.md;docs/architecture/security.md(trusted origins, keyless loopback, text wires)docs/CHANGELOG.mdhas entries under[Unreleased]AGENTS.md— n/a (no package, root script or convention changed)