Repository navigation
LOU-K3: support a real LLM provider on Cloudflare Worker deploy target - #15
Merged
Merged
Conversation
The cloudflare-worker deploy target only registered the 'mock' LLM
provider, per runtime.worker.ts's own comment because "the real provider
classes depend on optional-peer SDK packages that this bundle does not
include" - not because of an actual Workers incompatibility. Verified
both OpenAIProvider and AnthropicProvider (via @ai-sdk/openai and
@ai-sdk/anthropic on top of the Vercel `ai` SDK's fetch()-based
generateText/streamText) have zero `node:*` imports anywhere in their
dependency graphs, so both are genuinely Workers-compatible. Wire both
into runtime.worker.ts's LLMProviderRegistry and WORKER_SUPPORTED_PROVIDERS,
add @ai-sdk/openai as a devDependency alongside the existing
@ai-sdk/anthropic so both can be verified/bundled, and extend
cloudflare.test.ts to prove scaffold+build succeeds and produces a
zero-node:-references bundle for both, while a genuinely unsupported
provider ('ollama', which defaults to an unreachable local endpoint)
still fails closed with a clear error.
The 'http' built-in tool stays unsupported on Workers: its SSRF
protection depends on node:dns to resolve-then-verify every address
before connecting (closing a DNS-rebinding gap) and a pinned undici
Agent for per-request TLS settings, neither of which has a Workers-native
fetch() equivalent - reimplementing it on plain fetch() would silently
drop that protection rather than losing mere convenience, so it's left
unsupported rather than shipped weaker under the same name. Documented
in docs/deployment.md and runtime.worker.ts.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
cloudflare-workerdeploy target (src/deploy/adapters/cloudflare.ts,src/deploy/runtime.worker.ts) only supported themockLLM provider. Per the runtime's own doc comment, that was because "the real provider classes depend on optional-peer SDK packages that this bundle does not include" — not because of an actual Workers incompatibility. This closes that gap for two providers, and documents precisely what's still genuinely unsupported and why.What was verified
src/providers/OpenAIProvider.tsandsrc/providers/AnthropicProvider.ts: both are implemented on the VercelaiSDK'sgenerateText/streamTextplus@ai-sdk/openai/@ai-sdk/anthropic.@ai-sdk/anthropicwas already a devDependency at^0.0.42; added@ai-sdk/openai@0.0.42to match) and grepped their builtdist/*.js/*.mjsoutput, plus their dependencies (@ai-sdk/provider,@ai-sdk/provider-utils,ai,@ai-sdk/ui-utils) for anynode:*specifier — zero matches anywhere in either dependency graph. Both are purefetch()/Web-standard implementations. (ai's own devDependencies even include@edge-runtime/vm, confirming it's built with edge/Workers runtimes in mind.)What was added
src/deploy/runtime.worker.ts: registersopenaiandanthropicdirectly (not via the../providersbarrel, which would also eagerly pull inOllamaProvider/OpenRouterProviderand their peer SDKs) into the Worker'sLLMProviderRegistry.src/deploy/adapters/cloudflare.ts:WORKER_SUPPORTED_PROVIDERSis now['mock', 'openai', 'anthropic'].package.json: added@ai-sdk/openaias a devDependency (matching@ai-sdk/anthropic's pinned0.0.42) so both can actually be verified/bundled in this repo's own test/build.src/deploy/adapters/cloudflare.test.ts:ollamainstead ofopenai(sinceopenaiis now supported), and added an assertion on the fullWORKER_SUPPORTED_PROVIDERSlist.LLMProviderRegistryand thatprepareWorkerSpecresolves them (API key read from Workerenvbindings, notprocess.env).it.each(['openai', 'anthropic'])test: realscaffold()+build()(via tsup, platformbrowser) for each provider, asserting the built bundle has zeronode:references, then drives the built bundle's realfetch()handler end-to-end and asserts the/chatfailure is a genuine network/auth error (proving the provider was resolved and attempted a realfetch()call), not a "provider not found"/"not supported" error (which would mean the wiring itself was broken).docs/deployment.md: updated the Workers-support list and provider API key instructions.What is still genuinely unsupported, and why (precise, for the audit doc)
ollamaandopenrouterproviders: not wired in.ollama-ai-providerdefaults to a localhttp://localhost:11434endpoint a Cloudflare Worker cannot reach, so it isn't a realistic Workers target regardless of its dependency graph.openrouterhas had no Workers-compatibility audit in this change. Both remainnode-server/docker-only.httpbuilt-in tool: deliberately not reimplemented for Workers, per the task's own guidance to skip rather than ship something weaker.src/tools/built-in/http.ts's SSRF protection resolves the hostname vianode:dnsand checks every resolved address against a denylist before connecting (closing a DNS-rebinding gap), then pins per-request TLS behavior via a dedicatedundiciAgent/dispatcher. Cloudflare Workers' nativefetch()gives no hook to resolve a hostname up front and pin the underlying connection to the verified IP — a Workers version built on plainfetch()would only be able to check the literal hostname string, silently reintroducing the exact DNS-rebinding SSRF hole this tool was hardened against. That's a security regression, not a "reasonably contained" reimplementation, so it was skipped. Stillnode-server/docker-only.Test plan
npm run typecheck— clean, no errors.npm run lint— 0 errors introduced (5 pre-existingno-require-importserrors remain in unrelated, untouchedsrc/execution/__fixtures__/sample-repo/*.jsfixture files).npx vitest run src/deploy— 23 passed, 1 skipped (thewrangler-CLI-dependentdescribe.skipIfblock, gracefully skipped per the existing LOU-I pattern sincewrangler dev/deploy --dry-runintegration tests aren't runnable in this sandbox).🤖 Generated with Claude Code