[rileydes-improving]

• Add integration test validating all Redis command patterns against Valkey
• Add GitHub Actions workflow running Jawn tests against valkey:8-alpine
• Add self-hosting documentation for Valkey as a supported backend
• Zero application code changes; ioredis connects to Valkey transparently

Ticket

Helicone#5753

Component/Service

What part of Helicone does this affect?

• Web (Frontend)
• [ x ] Jawn (Backend)
• Worker (Proxy)
• Bifrost (Marketing)
• AI Gateway
• Packages
• [ x ] Infrastructure/Docker
• [ x ] Documentation

Type of Change

• Bug fix
• [ x ] New feature
• Breaking change
• [ x ] Documentation update
• Performance improvement
• Refactoring

Deployment Notes

• [ x ] No special deployment steps required
• Database migrations need to run
• Environment variable changes required
• Coordination with other teams needed

No application code is modified. Operators already on Valkey continue working as-is. Operators on Redis are completely unaffected. For new Valkey adopters, the only change is pointing REDIS_HOST at their Valkey endpoint.

Extra Notes

• Jawn's existing CI already exercises the graceful-degradation (no-Redis) path implicitly. This PR adds coverage for the Valkey-present path.

Context

Helicone's Redis dependency in Jawn works with Valkey transparently (all commands predate the fork), but this compatibility is undocumented, untested in CI, and invisible to operators evaluating Helicone's stack. Two of three closest competitors (LangFuse, LangSmith) already officially support Valkey. This PR adds tests, CI, and documentation confirming what already works.

The five subsystems that touch Redis (KV cache, proxy rate limiter, HTTP API rate limiter, usage-limit cache, encrypted provider key/auth cache) all share a single ioredis client and issue only core Redis commands present since Redis 2.x, all fully supported in Valkey.

[daric93]

Code Review: feat(jawn): add official Valkey backend support with CI and docs

Thanks for this — it's a clean, low-risk contribution, and adding CI + docs to certify existing compatibility is genuinely useful for operators evaluating the stack. My main concern is that the docs and CI over-promise relative to what the code and tests actually deliver.

What works well

• Zero production code change is the right call — Redis operators are completely unaffected, and the compatibility claim stands on its own.
• ~330 lines of well-structured tests with good edge-case coverage on the parts they do test: TTL expiry, LRU-style eviction, AES-GCM round-trip, random-IV non-determinism, corrupted-ciphertext rejection, and the "errors aren't cached" path in getAndStoreInCache.
• describeIfRedis gating cleanly separates pure-unit sections from those needing a live instance.

Issues requiring changes

Bugs / Docs (blocking):

1. Docs promise "just set REDIS_HOST, no other config," but the production client hardcodes tls: {} with no toggle — the plain-Valkey docker-compose example in the same doc would fail the TLS handshake. The test even mocks around this exact fact.
2. "What's Verified — tested against Valkey in CI" lists 5 subsystems; the test only exercises 2. The two rate limiters and the Usage Limit Cache are never imported by any test.

Test coverage:
3. The commands most worth proving (Lua SCRIPT LOAD/EVALSHA/PEXPIRE, the EX sliding window) are exactly the untested ones — a GET/SET round-trip has been Valkey-safe since the fork.

Design / Test fidelity:
5. CI certifies a hand-rolled ioredis client (conditional TLS, custom options), not the production singleton — so "drop-in compatible" is proven for a client prod never uses.
6. CI pins Valkey to the moving latest tag, disagreeing with the PR's valkey:8-alpine and the docs' "Valkey 8 or later."

Nits:
4. "serves from memory ... no extra Redis call" never asserts the second get skips Redis.
7. "Valkey 8 or later" is stricter than any listed command requires (all exist in 7.2).

Recommendations

• Fix #1 and Helicone#2 before merge — for a docs+CI PR, doc accuracy is the deliverable, and both describe behavior that doesn't hold (a connection path that fails, coverage that doesn't exist).
• Helicone#3, Helicone#5, Helicone#6 strengthen the compatibility guarantee but aren't hard blockers.
• Helicone#4 and Helicone#7 are minor.

See inline comments for detailed explanations and suggested fixes.

[MatthiasHowellYopp]

Review: Request Changes

⚠️ CI status: claude-review FAIL, Jawn Valkey Compatibility Tests PENDING. Other checks pass.

7 warnings, 2 nits. Two blocking themes:

1. Unpinned image tag (findings 1, 5, 7) — undermines reproducibility of the test signal this workflow provides
2. Docker Compose race (finding 4) — operators following the guide will hit intermittent startup failures

The PR is well-scoped (zero application code changes) and the test coverage is thorough. Addressing the warnings would make this ready to merge.

[Jonathan-Improving]

QA Review — PR #1: Official Valkey Backend Support

Verdict: COMMENT (no blocking issues — 8 Medium suggestions, 2 Low suggestions)

Summary

Solid PR. The CI correctly validates TLS connectivity against Valkey 7.2. Tests meaningfully exercise production code for 3 of 5 subsystems and accurately test the command surface for the other 2. Documentation is well-structured with an honest "What's Verified" table.

Key findings (8 Medium):

• Docs overclaim ("work identically" vs. "commands verified for 2 subsystems")
• Flaky 30ms/100ms timing margins in CI assertions
• KEYS command in cleanup (O(n) + copy-paste risk pattern)
• 18x non-null assertions without runtime guard
• CI actions not SHA-pinned
• No job-level timeout
• God test file (560 lines, 7 describe blocks, shared singleton)
• chmod 644 on private key (should be 600 for key, 644 for cert)

Suggestions (2 Low):

• --forceExit masks potential connection leaks
• describeIfRedis skips 4 suites silently with no output

Positives

• Production singleton used (no mocking) — validates real TLS connection behavior
• describeIfRedis pattern correctly gates integration tests while allowing unit tests to run freely
• TLS-only Valkey config (--port 0) matches production behavior
• NODE_EXTRA_CA_CERTS is the correct trust mechanism (no app code modification needed)
• Honest "What's Verified" table distinguishing subsystem-imported from command-verified
• Proper afterAll cleanup with REDIS_HOST guard
• Random suffixes per describe block prevent cross-test interference