[joshfactorial]

Closes #127. Part of #128.

Adds POST /api/public/v1/deployment/{deploymentId}/responses, the OpenAI Responses API, on the public API only (not the chat page). It reuses #137's shared proxyPublicDeploymentEndpoint, so it gets the same API-key checks as chat completions and embeddings, sends the deployment's own vLLM key upstream (#147), and passes the reply through, streamed or not.

What's supported

It's stateless (see the analysis in #127):

• ✅ Requests with input and instructions, sampling settings, and structured output
• ✅ Streaming ("stream": true): vLLM's Responses event stream passes through unchanged
• ✅ Multi-turn conversations, by sending the conversation again (input as a list of messages)
• ✅ Function tools, with the same model-side requirements as chat completions
• ❌ Anything that needs vLLM to remember responses: previous_response_id (vLLM returns 404), background, and retrieving or cancelling a stored response. store: true is ignored. Turning on vLLM's response store would need a server setting, has memory and per-user drawbacks, and would be a separate issue under Add missing API endpoints #128 if anyone needs it.
• ❌ Built-in tools (web search, code interpreter, MCP): vLLM supports them only for gpt-oss models with a tool server.

Changes

• frontend/app/api/public/v1/deployment/[deploymentId]/responses/route.ts: the new route, a thin wrapper like the embeddings one. Its header comment explains the stateless limits.
• frontend/lib/public-deployment-proxy.ts: 'responses' added to PublicDeploymentEndpoint. The security note now states that the caller's LLMHub key is never forwarded; only the deployment's vLLM key is.
• frontend/tests/unit/public-responses-route.test.ts: 10 tests.
• CHANGELOG.md: an entry under [Unreleased] → Added.

Testing

Automated: vitest 110 passed (10 new), tsc clean, eslint no errors. The new tests cover the same auth and proxy cases as the embeddings route, plus:

• the deployment's vLLM key is sent and the caller's llmhub_ key isn't;
• an SSE stream passes through unchanged;
• vLLM's 404 for previous_response_id reaches the caller (the mock uses vLLM's real error body, from _make_not_found_error).

Live on staging, 2026-11-04 (Delta, impersonate mode; staging on main at 0b1bcb7 with this branch's changes, frontend rebuilt). Qwen2.5-0.5B-Instruct deployment 7c619a0e, launched from the UI, called through the new route with an llmhub_ key:

Case	Result
Plain request	completed: "Hello! How can I assist you today?"
"stream": true	Full event stream: response.created, in_progress, 14 × output_text.delta, …, response.completed
Multi-turn, by sending the conversation again	completed: "Your name is Ada."
previous_response_id	404, vLLM's "Response with id '…' not found.", passed through
Wrong LLMHub key	401 from LLMHub; the request never reaches vLLM

The successful requests also confirm the deployment's vLLM key is sent: the deployment was launched with one, and vLLM refuses requests without it.

Not covered:

• DeltaAI: not run there. vLLM 0.28 (DeltaAI) has the same Responses routes and stateless behavior as 0.19.1 (checked in its source, vllm/entrypoints/openai/responses/), and this change doesn't depend on the cluster. A quick curl on DeltaAI dev after deploy would confirm it.
• gpt-oss models and their built-in tools.
• An API dialog example for this endpoint: the issue is API-only. It could be added later through 150 UI embeddings model #152's snippet table.

How to test

1. Unit tests, from frontend/:

pnpm test tests/unit/public-responses-route.test.ts

2. Deploy the branch to staging (Delta VM, as the service account; check nobody else is using staging first):

K=/projects/bfmz/svcdeltallmhub/llmhub-staging/LLMHub/infra/delta/llmhub
$K deploy --profile staging --ref 127-responses-endpoint --apply

3. Launch a small model (e.g. Qwen2.5-0.5B-Instruct) from the UI, wait for Running, and copy its deployment ID and an API key from the API dialog. Then, with U=http://127.0.0.1:3100/api/public/v1/deployment/<deployment id>/responses and KEY=llmhub_…:

# plain: expect "status": "completed" and a reply
curl -s $U -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"Qwen2.5-0.5B-Instruct","input":"Say hi","max_output_tokens":20}' | jq '{status, output}'

# streamed: expect response.created … response.output_text.delta … response.completed
curl -s -N $U -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"Qwen2.5-0.5B-Instruct","input":"Count to three","stream":true}' | grep '^event:'

# multi-turn by resending: expect it to answer "Ada"
curl -s $U -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"Qwen2.5-0.5B-Instruct","input":[{"role":"user","content":"My name is Ada."},{"role":"assistant","content":"Nice to meet you, Ada."},{"role":"user","content":"What is my name?"}]}' | jq '.output'

# stateless: expect 404 "Response with id ... not found."
curl -s -w '\nHTTP %{http_code}\n' $U -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
  -d '{"model":"Qwen2.5-0.5B-Instruct","input":"again","previous_response_id":"resp_x"}'

🤖 Generated with Claude Code