First-release polish (spec 020)

.claude/skills/rag-pipeline/SKILL.md

@@ -88,6 +88,7 @@ Spec 012 is the contract; this skill says where things live and what not to brea
 ## Persistence
 
 - `CitationSnapshot` copies the passage (ids, version, title, section, pages, score, **full text**). It is never updated, and nothing references the `documents` schema.
+- Spec 020: the snapshot also stores `TextHash` (`PassageTextHash.Compute` of its text, the same hash Documents keeps on every passage). Reading a conversation asks `IDocumentSources` once for all its citations and returns a `sourceState` per citation (current, newer-unchanged, newer-changed, archived, unavailable); a live answer cites the active versions, so its citations are `current`. Change the hash rule only together with both migrations' SQL backfill (`PassageTextHash.SqlExpression` and `CitationSourceStateTests` check they agree).
 - `RequestTrace` holds decision data only: question text, filters, outcome and thresholds, each kept passage (`passageId`, `versionId`, `score`, `rank`, `marker`, `cited`) as jsonb, models, prompt version, tokens, sentinel, stripped count, status, timings, OTel trace id. **No passage text and no prompt (ADR 0004 D4).**
 
 ## Providers

CLAUDE.md

@@ -12,8 +12,9 @@ Stack decisions are recorded in [ADR 0001](docs/adr/0001-ui-ai-streaming-stack.m
 
 | Module | Owns | Schema |
 |--------|------|--------|
-| `Documents` | Upload/ingest documents, text extraction, chunking, embedding generation, vector storage, similarity search. Exists: data model, `POST /api/documents`, `GET /api/documents/{id}` ([spec 006](docs/specs/006-documents-data-model-and-import.md)); background ingestion (`IngestionWorker`, chunker, batched embedding), `POST /api/documents/{id}/ingestion/retry` and the `GET …/ingestion/events` SSE stream ([spec 007](docs/specs/007-background-ingestion-and-progress.md)); listing with filters and paging, facets, versions and passages of a version (`GET /api/documents`, `/facets`, `/{id}/versions`, `/{id}/versions/{n}/passages`) and the Documents UI ([spec 008](docs/specs/008-browse-corpus-and-documents-ui.md)); edit metadata (`PUT`, `If-Match`/`ETag`), new version, reprocess, archive and restore, with the matching detail-page actions ([spec 009](docs/specs/009-document-lifecycle.md)); PDF import up to 100 MiB (streamed upload into a content-addressed file store, text-layer extraction with page numbers on passages, scanned PDFs rejected at upload), `GET …/versions/{n}/file` for the original and `progress` events inside long ingestion stages ([spec 010](docs/specs/010-pdf-import.md)); retrieval through `IDocumentSearch` (vector search over active, indexed versions with language and tag filters, evidence thresholds, near-duplicate reduction, cached query embeddings) and the maintainers' diagnostic `GET /api/documents/search` ([spec 011](docs/specs/011-retrieval.md)); private and shared documents: `Visibility` on `Document`, owner = `CreatedBy`, one `ReadableBy(viewerSub)` rule applied on every read path, so a document the caller cannot read is a 404 and `IDocumentSearch`/`IDocumentCatalog` take a required `viewerSub` ([spec 016](docs/specs/016-private-and-shared-documents.md)); maintainers' read side under `/api/admin/documents` (corpus health, currently failed versions, a metadata-only list of private documents that deliberately bypasses `ReadableBy`, configuration) and `IDocumentTitles`, which returns titles of shared documents only ([spec 017](docs/specs/017-diagnose-traces-admin-and-metrics.md)); the document title leads every embedding input (`EmbeddingInput`), and each version records `EmbeddingInputVersion`: a version embedded the old way stays searchable, counts as `outdatedEmbeddings` in admin health and is reprocessed by the harness `seed` ([spec 021](docs/specs/021-context-budget-title-embeddings-nightly-evaluation.md)); raise `EmbeddingInput.Version` whenever the input changes; hybrid search and reranking behind switches ([spec 019](docs/specs/019-hybrid-search-and-reranking.md)): a keyword pool (`passages.search_vector`, generated from `search_config`, which database triggers keep equal to the document language: `english`, `documents.french_unaccent` or `simple`) fused with the vector pool by reciprocal rank fusion (`HybridRanking`), then optionally a cross-encoder through `IPassageReranker` (`PassageReranking`: one call, its own timeout, any failure means `rerank: skipped` and cosine gating); every passage carries `PassageStages`, the trace and the admin inspector show them, and `GET /api/documents/search` takes `mode`, `rerank` and `queryLanguage` overrides | `documents` |
-| `Chat` | Conversations and messages, prompt building, retrieval orchestration, LLM answer generation with citations. Exists: owned conversations (`POST/GET /api/chat/conversations`, `GET …/{id}`), the SSE answer stream `POST …/{id}/messages` (refusal without a model call on `NoEvidence`, one prompt builder, `[n]` citations validated in the stream, sentinel veto, complete/partial status, citation snapshots), `POST …/messages/{id}/retry`, the stale-answer sweep and the write side of request traces ([spec 012](docs/specs/012-chat-conversations-and-grounded-answers.md)); follow-up questions (a bounded history window feeds a `GetResponseAsync<RewriteResult>` rewrite into a standalone retrieval query, with fallback to the original question), ratings and issue reports on finished answers, and hard deletion of a conversation with its traces ([spec 014](docs/specs/014-follow-ups-feedback-and-deletion.md)); a per-conversation scope of documents and tags (`PUT …/scope`, narrowing retrieval through `SearchFilters.DocumentIds`) and the stored activity of each answer (`GET …/messages/{id}/activity`) ([spec 015](docs/specs/015-chat-scope-activity-and-layout.md)); maintainers' read side under `/api/admin/chat` (requests and the request inspector's data, summary with SQL percentiles, feedback list, configuration) and the `RagAssistant.Chat` meter, recorded from the finished trace ([spec 017](docs/specs/017-diagnose-traces-admin-and-metrics.md)); a context budget (`PromptBudget`, `Chat:Generation:ContextWindowTokens`, sent to Ollama as `num_ctx` by the API's `OllamaRequestOptions`) drops the lowest-ranked passages before `retrieval.completed` and records the trim and the prompt estimate in the trace ([spec 021](docs/specs/021-context-budget-title-embeddings-nightly-evaluation.md)); load the `rag-pipeline` skill before changing it | `chat` |
+| `Documents` | Upload/ingest documents, text extraction, chunking, embedding generation, vector storage, similarity search. Exists: data model, `POST /api/documents`, `GET /api/documents/{id}` ([spec 006](docs/specs/006-documents-data-model-and-import.md)); background ingestion (`IngestionWorker`, chunker, batched embedding), `POST /api/documents/{id}/ingestion/retry` and the `GET …/ingestion/events` SSE stream ([spec 007](docs/specs/007-background-ingestion-and-progress.md)); listing with filters and paging, facets, versions and passages of a version (`GET /api/documents`, `/facets`, `/{id}/versions`, `/{id}/versions/{n}/passages`) and the Documents UI ([spec 008](docs/specs/008-browse-corpus-and-documents-ui.md)); edit metadata (`PUT`, `If-Match`/`ETag`), new version, reprocess, archive and restore, with the matching detail-page actions ([spec 009](docs/specs/009-document-lifecycle.md)); PDF import up to 100 MiB (streamed upload into a content-addressed file store, text-layer extraction with page numbers on passages, scanned PDFs rejected at upload), `GET …/versions/{n}/file` for the original and `progress` events inside long ingestion stages ([spec 010](docs/specs/010-pdf-import.md)); retrieval through `IDocumentSearch` (vector search over active, indexed versions with language and tag filters, evidence thresholds, near-duplicate reduction, cached query embeddings) and the maintainers' diagnostic `GET /api/documents/search` ([spec 011](docs/specs/011-retrieval.md)); private and shared documents: `Visibility` on `Document`, owner = `CreatedBy`, one `ReadableBy(viewerSub)` rule applied on every read path, so a document the caller cannot read is a 404 and `IDocumentSearch`/`IDocumentCatalog` take a required `viewerSub` ([spec 016](docs/specs/016-private-and-shared-documents.md)); maintainers' read side under `/api/admin/documents` (corpus health, currently failed versions, a metadata-only list of private documents that deliberately bypasses `ReadableBy`, configuration) and `IDocumentTitles`, which returns titles of shared documents only ([spec 017](docs/specs/017-diagnose-traces-admin-and-metrics.md)); the document title leads every embedding input (`EmbeddingInput`), and each version records `EmbeddingInputVersion`: a version embedded the old way stays searchable, counts as `outdatedEmbeddings` in admin health and is reprocessed by the harness `seed` ([spec 021](docs/specs/021-context-budget-title-embeddings-nightly-evaluation.md)); raise `EmbeddingInput.Version` whenever the input changes; hybrid search and reranking behind switches ([spec 019](docs/specs/019-hybrid-search-and-reranking.md)): a keyword pool (`passages.search_vector`, generated from `search_config`, which database triggers keep equal to the document language: `english`, `documents.french_unaccent` or `simple`; the question's lexemes are OR-ed, and a match holding at least `Hybrid:MinKeywordCoverage` of them passes the cosine floor) fused with the vector pool by reciprocal rank fusion (`HybridRanking`), then optionally a cross-encoder through `IPassageReranker` (`PassageReranking`: one call, its own timeout, any failure means `rerank: skipped` and cosine gating); every passage carries `PassageStages`, the trace and the admin inspector show them, and `GET /api/documents/search` takes `mode`, `rerank` and `queryLanguage` overrides; first-release polish ([spec 020](docs/specs/020-first-release-polish.md)): after indexing, `SuggestionGenerator` asks the chat model (`IChatClient`, structured output) for 3 questions and keeps those retrieval answers confidently as the owner, on the version (`SuggestedQuestions`, `SuggestionsStatus` pending/ready/failed/skipped/edited); it never fails the run, is off in demo mode (`Documents:Suggestions:Enabled`, turned off by the API) and records a `suggestions` stage event after `completed`; the owner edits them with `PUT …/{id}/suggestions` (`If-Match`), edited ones survive reprocess; `GET /api/documents/suggestions` and `/summary` feed the home page; every passage stores `TextHash` (`PassageTextHash` in the kernel, also written out in the migrations' SQL) and `IDocumentSources` gives each cited passage a source state (current, newer-unchanged, newer-changed, archived, unavailable) | `documents` |
+| `Chat` | Conversations and messages, prompt building, retrieval orchestration, LLM answer generation with citations. Exists: owned conversations (`POST/GET /api/chat/conversations`, `GET …/{id}`), the SSE answer stream `POST …/{id}/messages` (refusal without a model call on `NoEvidence`, one prompt builder, `[n]` citations validated in the stream, sentinel veto, complete/partial status, citation snapshots), `POST …/messages/{id}/retry`, the stale-answer sweep and the write side of request traces ([spec 012](docs/specs/012-chat-conversations-and-grounded-answers.md)); follow-up questions (a bounded history window feeds a `GetResponseAsync<RewriteResult>` rewrite into a standalone retrieval query, with fallback to the original question), ratings and issue reports on finished answers, and hard deletion of a conversation with its traces ([spec 014](docs/specs/014-follow-ups-feedback-and-deletion.md)); a per-conversation scope of documents and tags (`PUT …/scope`, narrowing retrieval through `SearchFilters.DocumentIds`) and the stored activity of each answer (`GET …/messages/{id}/activity`) ([spec 015](docs/specs/015-chat-scope-activity-and-layout.md)); maintainers' read side under `/api/admin/chat` (requests and the request inspector's data, summary with SQL percentiles, feedback list, configuration) and the `RagAssistant.Chat` meter, recorded from the finished trace ([spec 017](docs/specs/017-diagnose-traces-admin-and-metrics.md)); a context budget (`PromptBudget`, `Chat:Generation:ContextWindowTokens`, sent to Ollama as `num_ctx` by the API's `OllamaRequestOptions`) drops the lowest-ranked passages before `retrieval.completed` and records the trim and the prompt estimate in the trace ([spec 021](docs/specs/021-context-budget-title-embeddings-nightly-evaluation.md)); conversations can be renamed (`PATCH …/{id}`, `TitleEdited`; an empty title reverts to the first question), searched (`q`, `unaccent` + `ILIKE` over titles and questions) and exported as Markdown (`GET …/{id}/export`, the source lines of `SourceList` match the client's "Copy answer with sources" in `SourceListText`), and citations carry a `sourceState` computed from `CitationSnapshot.TextHash` when the conversation is read ([spec 020](docs/specs/020-first-release-polish.md)); load the `rag-pipeline` skill before changing it | `chat` |
+| `Evaluation` | Harness reports uploaded for maintainers ([spec 020](docs/specs/020-first-release-polish.md), roadmap 5.3): `POST /api/admin/evaluation/runs` (report JSON ≤ 10 MiB, schema 1, `sample` scope only, idempotent by run id, an optional `judge` property replaces the judged metrics), list, detail, compare and delete under `/api/admin/evaluation`, Maintainer only. It reads the report as JSON (`Common/Json.cs`) and binds none of the harness's types; no contract with other modules | `evaluation` |
 
 Modules are added with `/scaffold`. In each module's `AddMediator(...)` set `GenerateTypesAsInternal = true` and `Assemblies = [typeof(<Module>Module)]`: the default scans every referenced assembly and generates public helper types, which collide with other modules and with the test projects' own Mediator (ADR 0005). A module reads its connection string itself (`UseNpgsql(connectionString, o => o.UseVector())`) and ships an `IDesignTimeDbContextFactory` so `dotnet ef` does not need to start the API.
 
@@ -238,6 +239,8 @@ dotnet run --project tools/RagAssistant.Evaluation -- seed
 dotnet run --project tools/RagAssistant.Evaluation -- run
 # ... and also measure retrieval for vector, hybrid and hybrid + rerank on the same queries (spec 019)
 dotnet run --project tools/RagAssistant.Evaluation -- run --variants
+# run and judge upload their report to the app's admin area when done (spec 020); --no-upload keeps it local, upload sends an older one
+dotnet run --project tools/RagAssistant.Evaluation -- upload .data/evaluation/<run>
 # The same run happens nightly in CI (.github/workflows/evaluation.yml, spec 021); trigger it by hand with
 gh workflow run evaluation.yml
 # Offline on reports: diff two runs, sweep retrieval thresholds; judge needs Ollama (pulls Evaluation:JudgeModel on demand)