feat: replace Elasticsearch with PostgreSQL full-text search (PPT-2644, 2/2) - #443
Closed
camreeves wants to merge 3 commits into
Closed
feat: replace Elasticsearch with PostgreSQL full-text search (PPT-2644, 2/2)#443camreeves wants to merge 3 commits into
camreeves wants to merge 3 commits into
Conversation
…on (PPT-2644) - Utils::TextSearch: never-500 q -> tsquery sanitizer (AND of prefix tokens) - application.cr: search_tsquery/search_limit/search_offset + paginate_search; fix the Link header off-by-one that skipped one row per page - modules: parent-driver search via EXISTS join (has_parent parity); scoped listing now uses IN() (the ES version returned nothing for >1 visible module) - zones: tree/tag filters in SQL, children counts via GROUP BY - drivers: straight conversion NOTE: pg-orm 2.2.x Collections cannot compile a single-argument raw where() (NamedTuple-in-union overload) — always pass bind args. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…PT-2644) - docker-compose: remove elastic + search-ingest services/anchors - spec/helper: remove refresh_elastic/ensure_keyword_mapping/clear_elastic - test_base_index macro: synchronous save->query->assert (no index lag) - root.cr: /reindex + /backfill deprecated no-ops; search_ingest out of the cluster version fan-out; require removed - shard.yml/lock: search-ingest dependency removed (neuroplastic remains transitively via placeos-models until the models major drops it) - spec/migration: TEMPORARY models branch pin (revert before merge) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…2644) All 24 Elasticsearch-backed controllers now query PostgreSQL directly: filters become SQL WHERE clauses, q matches the generated search_vector columns, and ES parent-child becomes EXISTS joins (modules<->driver, system-triggers<->trigger, systems<->trigger instances, assets<->asset_type). paginate_results and the ES spec helpers are gone. Deliberate behavior fixes (each spec-pinned): - systems?trigger_id= worked never (unsatisfiable ES query) — real join - systems?email= was a no-op alone and WIDENED the group/subsystem zone scope when combined — now a strict AND filter - scoped (non-admin) module listing returned [] when the user could access more than one module — now IN() - alerts multi-dashboard scoping matched nothing for authorities with more than one dashboard — now IN() - alerts?enabled=false and asset_categories?hidden=false were ignored (falsy-Bool) — now filter both values - system-triggers q was silently ignored — now matches the parent trigger's text - assets can now be searched by asset-type name (previously a TODO) Known ES-era quirks preserved: zones/assets/pending_mails multi-zone AND-containment, public/important/triggered only-when-true filters, undecrypted settings search results. pg-orm gotchas encoded: never single-arg raw where (does not compile); arrays cannot bind as one parameter — sql_array() splices per-element placeholders; empty arrays guard with an empty-page short-circuit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Opened prematurely — closed pending internal review. Work continues on the branch (PPT-2644).