Repository navigation
fix(gateway): Gaia's 16 Oct issues, DAG barge-in and streaming TTS, realtime close, build and test fixes - #19
Merged
Conversation
…ealtime close, build and test fixes
Reported issues
- DAG TTS never delivered: select! dropped the completion receiver; it is
now polled in place, and DAG failures are typed errors, not just logs.
- LiveKit room calls 404: livekit-api encoded the path's slashes, which
LiveKit 1.13 rejects; vendored livekit-api sends a literal path.
- ${VAR} sent literally: named credentials, resolved per LLM leg.
- Inline DAG could send the operator key anywhere: credentials are bound to
hosts; SSRF and DNS guards on every outbound DAG call.
- Debug logs printed API keys: redacting Debug; spans skip secrets.
- The API secret disabled the auth hook: AUTH_MODE chain (api_secret or
responder).
- A responder answering 200 {} admitted callers: an identity is required,
otherwise 502.
- STT 5 s and LLM 60 s limits had no knobs: env and config knobs (the DAG
LLM knob is now honoured too).
- /metrics was public: MetricsPolicy (token, separate listener or 403).
- A parent-directory .env was read: ./.env or --env-file only.
- Segmented STT stayed off unless allowlisted: an explicit request opts in.
- The denoiser never ran on /ws: per-session streaming denoiser, opt-in,
warming up without blocking `ready`.
Follow-ups
- DAG turns run as their own task on the streaming executor: barge-in
cancels the turn and clears TTS, a newer turn supersedes an older one, and
node execution races the cancel token. The TTS node opens one provider per
turn, speaks each sentence as the LLM writes it and forwards audio chunk
by chunk (first audio ~1.6 s earlier).
- Webhook and HTTP DAG nodes use the connect-time guarded resolver (no DNS
rebinding).
- Transcripts are redacted from the stdout log unless WAAV_LOG_CONTENT=1.
- A refused credential or failed DAG / conversation / LiveKit setup ends
the session with a 1011 close instead of an STT-only session.
- Inline DAG definitions are linted for unknown keys, including in arrays.
- OpenAI TTS URL no longer doubles /v1.
- Realtime teardown keeps reading the client while the backlog and close go
out, so a slow client gets its remaining audio and the 1011
client_too_slow close instead of a TCP reset; the admission slot is
released at once.
Build and tests
- Vendored webrtc-sys builds the NVIDIA codec on x86 only, so a fresh
aarch64 build with CUDA installed succeeds; Cargo.lock regenerated.
- Realtime and voice-span tests generate their credential and JWT keys at
run time instead of reading bud-auth's git-ignored *.pem fixtures; the
breaker tests open their breaker after the gateway is up; the slow-client
test sends more than the kernel can buffer.
Tests: 644 / 644 targeted (DAG, security, realtime relay and translate,
voice spans, LiveKit, credentials, denoiser), with ORT_DYLIB_PATH set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
dittops
force-pushed
the
fix/gaia-integration-2026-10-16
branch
from
October 17, 2026 20:12
88cdc63 to
b83fdca
Compare
…runcated replies
- Segmented STT: the waav-infer file profile now sends the session language
(`language`, single shape, droppable on refusal) — the Gateway used to send
none, so Infer guessed the language per segment. Routing map, capability map
and the wire test updated.
- Native Infer STT/TTS adapters default to Infer's real endpoint,
ws://127.0.0.1:8080/v1/ws (was ws://127.0.0.1:8123/v1/realtime).
- TTS: a speech reply WaaV Infer flags with `x-waav-truncated: true` (its
length bound cut the utterance; the audio is real but incomplete) is logged
and counted in waav_gateway_tts_truncated_total{provider}, instead of being
passed off as a complete reply. Pairs with Infer's new `truncated` terminal.
Tests: segmented-stt 490/490; gateway lib 7110/7113 — the three failures
(core::metrics::provider test_ttfb_recording / test_concurrent_min_max_updates,
core::observability::speaking test_typical_speaking_session) are timing tests
in untouched code that pass on rerun (one intermittent in 5 stress runs).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
4 tasks done
…ws test All of these also failed on #19's head; main was last green on 5 Oct. - supply-chain: rustls 0.23.45 -> 0.23.46 (RUSTSEC-2026-0346..0349). The rest cannot be upgraded here and are ignored with reasons in deny.toml (mirrored in gateway/.cargo/audit.toml), like the existing tonic 0.11 entries: rustls 0.22 only via tonic 0.11 (Google Speech client); zip extract() advisories while WaaV never extracts (zip 0.6 is webrtc-sys-build's build-time unpack, zip 2 is ndarray-npy's .npz reader via mel_spec); unsafe-libyaml unmaintained via serde_yaml 0.9 (operator-authored YAML). `cargo audit --deny warnings` and `cargo deny check advisories bans licenses sources` pass. - clippy -D warnings (Rust 1.99): chunks_exact -> as_chunks, is_multiple_of, and an explained allow where a `return` is only needless without features. - server boot: config.example.yaml configures API secrets, so #19's metrics policy locks /metrics (403) — the secure default. The probe now asserts the 403, then reboots with WAAV_METRICS_PUBLIC=true and checks the series. - ws_tests::test_websocket_voice_config: #19 closes a session whose voice setup failed (1011) instead of leaving it half-configured; the test now asserts that close rather than writing into the closed socket. Verified: clippy (default and --all-features) clean; the CI integration suites (e2e_mock, load, server_startup, ws, keystone_wire, keystone_completeness, segmented_stt_ws, segmented_stt_golden) 63/63. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI's rustfmt and segmented-stt jobs failed on #19's head. Formatting only, no functional change (vendored webrtc-sys untouched); both crates build with all targets afterwards. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gateway/vendor/ (webrtc-sys, livekit-api, vendored in #19) is upstream code kept byte-close to its source; its spellings are not ours to change. With cargo audit and cargo deny now green, typos on vendor/ was the last supply-chain failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…10-19 feat(gateway): WaaV Infer integration — session language, endpoint, truncated replies
gitleaks flagged gateway/vendor/livekit-api/src/test_token.txt (rule jwt) once #20 merged into this branch and the scan covered the whole range to main. It is the upstream crate's own test token (iss "myapikey", sub "identity", expired March 2025), read only by its access_token.rs tests. Allowlisted by exact path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Root-cause fixes for every Gateway issue in Gaia's 16 Oct report, plus the follow-ups found while verifying them. Each item was checked against the same inputs as Gaia's original release build (mock vendors, a real waav-infer, an auth responder and a real LiveKit 1.13.10 server).
Reported issues
select!dropped the completion receiver; errors only logged${VAR}sent literallyDebugand tracing spans record keysDebug; spans skip secretsAUTH_MODEchain: api_secret or responder200 {}admits callers/metricsis public.envis read./.envor--env-fileonly/ws/wsnever called it; LiveKit frames too shortreadyFollow-ups
<redacted>in the stdout log unlessWAAV_LOG_CONTENT=1(OTLP export unchanged).nodes[1].voice_ID)./v1.CUDA_HOME/clang workarounds; Cargo.lock regenerated.*.pemfixtures. Two breaker tests now open their 30 s breaker after the gateway is up, and the slow-client test sends more than the kernel can buffer.Behaviour changes to note
WAAV_LOG_CONTENT=1restores them).Test plan
ORT_DYLIB_PATHset (turn-detector tests need ONNX Runtime)🤖 Generated with Claude Code