Skip to content

LOU-K3: support a real LLM provider on Cloudflare Worker deploy target - #15

Merged
LinuxDevil merged 1 commit into
mainfrom
loushy/lou-k3
Sep 28, 2026
Merged

LinuxDevil merged 1 commit into
mainfrom
loushy/lou-k3

Conversation

@LinuxDevil

Copy link
Copy Markdown
Owner

Summary

The cloudflare-worker deploy target (src/deploy/adapters/cloudflare.ts, src/deploy/runtime.worker.ts) only supported the mock LLM provider. Per the runtime's own doc comment, that was because "the real provider classes depend on optional-peer SDK packages that this bundle does not include" — not because of an actual Workers incompatibility. This closes that gap for two providers, and documents precisely what's still genuinely unsupported and why.

What was verified

  • Read src/providers/OpenAIProvider.ts and src/providers/AnthropicProvider.ts: both are implemented on the Vercel ai SDK's generateText/streamText plus @ai-sdk/openai/@ai-sdk/anthropic.
  • Installed both packages (@ai-sdk/anthropic was already a devDependency at ^0.0.42; added @ai-sdk/openai@0.0.42 to match) and grepped their built dist/*.js/*.mjs output, plus their dependencies (@ai-sdk/provider, @ai-sdk/provider-utils, ai, @ai-sdk/ui-utils) for any node:* specifier — zero matches anywhere in either dependency graph. Both are pure fetch()/Web-standard implementations. (ai's own devDependencies even include @edge-runtime/vm, confirming it's built with edge/Workers runtimes in mind.)

What was added

  • src/deploy/runtime.worker.ts: registers openai and anthropic directly (not via the ../providers barrel, which would also eagerly pull in OllamaProvider/OpenRouterProvider and their peer SDKs) into the Worker's LLMProviderRegistry.
  • src/deploy/adapters/cloudflare.ts: WORKER_SUPPORTED_PROVIDERS is now ['mock', 'openai', 'anthropic'].
  • package.json: added @ai-sdk/openai as a devDependency (matching @ai-sdk/anthropic's pinned 0.0.42) so both can actually be verified/bundled in this repo's own test/build.
  • src/deploy/adapters/cloudflare.test.ts:
    • Updated the "rejects unsupported provider" test to use ollama instead of openai (since openai is now supported), and added an assertion on the full WORKER_SUPPORTED_PROVIDERS list.
    • New test asserting both providers are registered in the Worker's LLMProviderRegistry and that prepareWorkerSpec resolves them (API key read from Worker env bindings, not process.env).
    • New it.each(['openai', 'anthropic']) test: real scaffold() + build() (via tsup, platform browser) for each provider, asserting the built bundle has zero node: references, then drives the built bundle's real fetch() handler end-to-end and asserts the /chat failure is a genuine network/auth error (proving the provider was resolved and attempted a real fetch() call), not a "provider not found"/"not supported" error (which would mean the wiring itself was broken).
  • docs/deployment.md: updated the Workers-support list and provider API key instructions.

What is still genuinely unsupported, and why (precise, for the audit doc)

  • ollama and openrouter providers: not wired in. ollama-ai-provider defaults to a local http://localhost:11434 endpoint a Cloudflare Worker cannot reach, so it isn't a realistic Workers target regardless of its dependency graph. openrouter has had no Workers-compatibility audit in this change. Both remain node-server/docker-only.
  • http built-in tool: deliberately not reimplemented for Workers, per the task's own guidance to skip rather than ship something weaker. src/tools/built-in/http.ts's SSRF protection resolves the hostname via node:dns and checks every resolved address against a denylist before connecting (closing a DNS-rebinding gap), then pins per-request TLS behavior via a dedicated undici Agent/dispatcher. Cloudflare Workers' native fetch() gives no hook to resolve a hostname up front and pin the underlying connection to the verified IP — a Workers version built on plain fetch() would only be able to check the literal hostname string, silently reintroducing the exact DNS-rebinding SSRF hole this tool was hardened against. That's a security regression, not a "reasonably contained" reimplementation, so it was skipped. Still node-server/docker-only.

Test plan

  • npm run typecheck — clean, no errors.
  • npm run lint — 0 errors introduced (5 pre-existing no-require-imports errors remain in unrelated, untouched src/execution/__fixtures__/sample-repo/*.js fixture files).
  • npx vitest run src/deploy — 23 passed, 1 skipped (the wrangler-CLI-dependent describe.skipIf block, gracefully skipped per the existing LOU-I pattern since wrangler dev/deploy --dry-run integration tests aren't runnable in this sandbox).

🤖 Generated with Claude Code

The cloudflare-worker deploy target only registered the 'mock' LLM
provider, per runtime.worker.ts's own comment because "the real provider
classes depend on optional-peer SDK packages that this bundle does not
include" - not because of an actual Workers incompatibility. Verified
both OpenAIProvider and AnthropicProvider (via @ai-sdk/openai and
@ai-sdk/anthropic on top of the Vercel `ai` SDK's fetch()-based
generateText/streamText) have zero `node:*` imports anywhere in their
dependency graphs, so both are genuinely Workers-compatible. Wire both
into runtime.worker.ts's LLMProviderRegistry and WORKER_SUPPORTED_PROVIDERS,
add @ai-sdk/openai as a devDependency alongside the existing
@ai-sdk/anthropic so both can be verified/bundled, and extend
cloudflare.test.ts to prove scaffold+build succeeds and produces a
zero-node:-references bundle for both, while a genuinely unsupported
provider ('ollama', which defaults to an unreachable local endpoint)
still fails closed with a clear error.

The 'http' built-in tool stays unsupported on Workers: its SSRF
protection depends on node:dns to resolve-then-verify every address
before connecting (closing a DNS-rebinding gap) and a pinned undici
Agent for per-request TLS settings, neither of which has a Workers-native
fetch() equivalent - reimplementing it on plain fetch() would silently
drop that protection rather than losing mere convenience, so it's left
unsupported rather than shipped weaker under the same name. Documented
in docs/deployment.md and runtime.worker.ts.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@LinuxDevil
LinuxDevil merged commit 8a328f0 into main Sep 28, 2026
1 check failed
@LinuxDevil
LinuxDevil deleted the loushy/lou-k3 branch September 28, 2026 23:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant