Repository navigation
fix(retrieval): OR keyword query and coverage-gated floor exemption (spec 019) - #21
Merged
YBazanPro merged 1 commit intoOct 20, 2026
Conversation
…ast the cosine floor (spec 019) websearch_to_tsquery ANDs every word, so a natural-language question matched almost no passage and hybrid search equalled vector search (baseline variants: R@5 0.86 / MRR 0.84 for both). The question's lexemes are now OR-ed into one tsquery, quoted as they are so nothing stems them twice. While cosine is the gate (reranking off or skipped), the floor also dropped every keyword-only hit below MinScore, the exact-term matches hybrid search exists for. A keyword match now passes the floor when the passage holds at least Hybrid:MinKeywordCoverage (default 0.5) of the question's distinct lexemes; one common word is not evidence. ConfidentScore still applies, so keyword-only evidence is low-confidence. Coverage is a new stage field on the diagnostic endpoint; the setting is shown in the admin configuration and recorded in harness reports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #20 (spec 020): this PR's base is
feat/spec-020-first-release-polish, because the change touches files spec 020 also changed.Problem
The hybrid keyword pool contributed nothing:
websearch_to_tsqueryANDs every word, so "How long do I have to submit an expense claim after a trip?" only matched a passage holding all ofclaim,expens,long,submit,trip. On the sample corpus, typical golden questions got 0 keyword hits.A baseline
run --variantson the current code confirms it: vector and hybrid are identical (R@5 0.86, MRR 0.84, behaviour 0.83, refusals 0.80).Change
tsquery, still ranked withts_rank_cd. They are quoted in as Postgres produced them, so nothing stems them a second time.Documents:Retrieval:Hybrid:MinKeywordCoverage(default 0.5) passes the cosine floor. One common word is not evidence: on the dev corpus, "Can I bring my dog to the office?" matches 6 passages through offic alone, at coverage 0.33.ConfidentScorestill applies, so keyword-only evidence isLowConfidence. When reranking is applied, the rerank score gates as before.KeywordCoverageis a new stage field onGET /api/documents/search. The setting shows in the admin configuration and in harness reports.Verification
dotnet build -c Release: 0 warnings.dotnet format --verify-no-changes: clean.HybridSearchTests14/14, plus the other retrieval classes (reranking, contract, endpoint, visibility, candidates, retrieval stages, suggested questions, TEI) 76/76. The full integration suite was not run locally; CI runs it.Not done yet
MinKeywordCoverage = 0.5is provisional until that run.KeywordCoverageis not yet in the chat request trace or the admin inspector, only on the diagnostic endpoint.🤖 Generated with Claude Code