Skip to content

fix(retrieval): OR keyword query and coverage-gated floor exemption (spec 019) - #21

Merged
YBazanPro merged 1 commit into
feat/spec-020-first-release-polishfrom
fix/spec-019-keyword-or-semantics
Oct 20, 2026
Merged

YBazanPro merged 1 commit into
feat/spec-020-first-release-polishfrom
fix/spec-019-keyword-or-semantics

Conversation

@YBazanPro

Copy link
Copy Markdown
Owner

Stacked on #20 (spec 020): this PR's base is feat/spec-020-first-release-polish, because the change touches files spec 020 also changed.

Problem

The hybrid keyword pool contributed nothing:

  • AND semantics: websearch_to_tsquery ANDs every word, so "How long do I have to submit an expense claim after a trip?" only matched a passage holding all of claim, expens, long, submit, trip. On the sample corpus, typical golden questions got 0 keyword hits.
  • Cosine floor: with reranking off or skipped, keyword-only hits still had to pass the cosine floor. It dropped exactly the exact-term matches hybrid search exists for.

A baseline run --variants on the current code confirms it: vector and hybrid are identical (R@5 0.86, MRR 0.84, behaviour 0.83, refusals 0.80).

Change

  • OR query: the question's distinct lexemes are OR-ed into one tsquery, still ranked with ts_rank_cd. They are quoted in as Postgres produced them, so nothing stems them a second time.
  • Coverage: each keyword match records the share of the question's lexemes the passage contains.
  • Floor exemption: while cosine is the gate, a match with coverage ≥ Documents:Retrieval:Hybrid:MinKeywordCoverage (default 0.5) passes the cosine floor. One common word is not evidence: on the dev corpus, "Can I bring my dog to the office?" matches 6 passages through offic alone, at coverage 0.33. ConfidentScore still applies, so keyword-only evidence is LowConfidence. When reranking is applied, the rerank score gates as before.
  • Visibility: KeywordCoverage is a new stage field on GET /api/documents/search. The setting shows in the admin configuration and in harness reports.
  • Docs: spec 019 (scope, evidence gate, edge cases, AC-23 to AC-25) and CLAUDE.md are updated.

Verification

  • dotnet build -c Release: 0 warnings. dotnet format --verify-no-changes: clean.
  • Documents unit tests: 305/305.
  • Integration: HybridSearchTests 14/14, plus the other retrieval classes (reranking, contract, endpoint, visibility, candidates, retrieval stages, suggested questions, TEI) 76/76. The full integration suite was not run locally; CI runs it.

Not done yet

  • The variants have not been rerun on this code, so there is no "after" number. The baseline run stopped before the hybrid + rerank variant. The calibration section of spec 019 (AC-20 to AC-22) is still open, and MinKeywordCoverage = 0.5 is provisional until that run.
  • KeywordCoverage is not yet in the chat request trace or the admin inspector, only on the diagnostic endpoint.

🤖 Generated with Claude Code

…ast the cosine floor (spec 019)

websearch_to_tsquery ANDs every word, so a natural-language question
matched almost no passage and hybrid search equalled vector search
(baseline variants: R@5 0.86 / MRR 0.84 for both). The question's
lexemes are now OR-ed into one tsquery, quoted as they are so nothing
stems them twice.

While cosine is the gate (reranking off or skipped), the floor also
dropped every keyword-only hit below MinScore, the exact-term matches
hybrid search exists for. A keyword match now passes the floor when the
passage holds at least Hybrid:MinKeywordCoverage (default 0.5) of the
question's distinct lexemes; one common word is not evidence.
ConfidentScore still applies, so keyword-only evidence is low-confidence.

Coverage is a new stage field on the diagnostic endpoint; the setting is
shown in the admin configuration and recorded in harness reports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@YBazanPro
YBazanPro merged commit d22336b into feat/spec-020-first-release-polish Oct 20, 2026
1 check passed
@YBazanPro
YBazanPro deleted the fix/spec-019-keyword-or-semantics branch October 20, 2026 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant