API: Always return usage for non-streaming completions

[blaakstone]

Is your pull request related to a problem? Please describe.
Non-streaming /v1/chat/completions and /v1/completions always return "usage": null.

Both non-streaming paths do

return_usage = data.stream_options and data.stream_options.include_usage

stream_options only exists for streaming requests, so this is always falsy there and the usage stats get dropped. Looks like a side effect of 1794791 ("Rework tool calls and OAI chat completions"): before it, the non-streaming response built usage=UsageStats(...) unconditionally, and the rework reused the streaming condition for both paths.

Why should this feature be added?
OpenAI returns usage on every non-streaming response. Clients that don't stream currently get no token counts, and proxies that read them (llama-swap's activity view here) show zeros.

Examples
Same greedy request, streaming with include_usage vs non-streaming, before and after:

endpoint	before	after	streaming
chat	null	25 / 108	25 / 108
completions	null	5 / 24	5 / 24

(prompt / completion tokens)

On the chat endpoint I also checked completion_tokens against /v1/token/encode on the output (108 vs 107, the extra one is the stop token), prompt_tokens against the encoded /v1/apply-template prompt (25 = 25), and that n=2 aggregates (214).

Additional context
Streaming behaviour is unchanged.

#478 has a test, test_chat_response_carries_timings_without_usage, that asserts usage is None when include_usage isn't set, so it pins the current behaviour and would need adjusting alongside this.

🤖 Generated with Claude Code