Skip to content

Index existing entries for search without depending on the database locale #2589

Open

Description

@pournasserian

Part of #2430 (F-SRC). Found by the review of #2566.

The PostgreSQL migration that indexes existing entries splits their text with regexp_replace(..., '[^[:alnum:]]+', ' ', 'g'). On a database whose LC_CTYPE is C, [:alnum:] is ASCII only, so every non-ASCII letter becomes a separator: "Café ÉLAN Ñandú" is stored as "caf lan and". Those entries can't be found by "café" until they are saved again. New writes are indexed by SearchText in C# and are not affected.

Scope

  • Index the entries that existed before the search tables with SearchText, the same code that indexes every write, instead of SQL that depends on the database's locale. For example, the migrations create the tables and a one-time step after migrating indexes every entry with no search document.
  • Keep the SQLite and PostgreSQL results the same for the same entries.

Out of scope

  • Changing how new writes are indexed.

Acceptance

  • On a PostgreSQL database created with LC_CTYPE=C, an entry with "Café" that existed before the search migration is found by "café" after the upgrade.
  • Entries that existed before the migration are found the same way on SQLite and PostgreSQL, and the migration's Down still works.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:infraPersistence, host, CI and buildtype:bugDefectv2FluentCMS v2 (v2 branch)

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions