Skip to content

Add configurable content name normalization with runtime Settings overrides - #2248

Merged
VargaJoe merged 9 commits into
developfrom
feature/content-naming-normalization
Nov 5, 2026
Merged

VargaJoe merged 9 commits into
developfrom
feature/content-naming-normalization

Conversation

@VargaJoe

@VargaJoe VargaJoe commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Generated content names used in public URLs need configurable normalization while preserving their human-readable DisplayName. This change adds four independent, disabled-by-default steps and lets operators disable configured steps through repository Settings without restarting the host.

Configuration and processing

All switches live under sensenet:contentNaming and default to false:

Switch Behavior
ApplyCustomReplacements Applies literal, case-sensitive ReplacementSources / ReplacementTargets string pairs in order.
RemoveDiacritics Removes decomposable Unicode diacritics.
Transliterate Uses AnyAscii 0.3.3 for Unicode-to-ASCII transliteration.
Lowercase Applies invariant lowercasing.

Processing order: custom replacements → diacritic removal → transliteration → lowercase → existing invalid-character replacement → original extension append.

Replacement arrays are paired by index, support multi-character strings and empty deletion targets, and may cascade in configured order. Non-empty sources and equal array lengths are required when the step is enabled. Use string arrays and contiguous zero-based environment indexes. The existing SnConfig scalar-list helper does not support JSON arrays, so the two new arrays read indexed configuration values through the existing GetString API.

The seven Docker webapps include 36 prepared replacement pairs in their base appsettings.json files, with all four switches explicitly false. This follow-up changes only JSON configuration and documentation; the configuration reader, naming pipeline and other C# code remain unchanged. There is no built-in fallback when host configuration omits the arrays.

The prepared list includes German-style ä → ae, ö → oe, ü → ue, ß → ss, uppercase counterparts (Ä → AE, Ö → OE, Ü → UE, ẞ → SS), other Latin letters, & → and, and spaces to hyphens:

{
  "sensenet": {
    "contentNaming": {
      "RemoveDiacritics": false,
      "Lowercase": false,
      "Transliterate": false,
      "ApplyCustomReplacements": false,
      "ReplacementSources": [
        "ä", "Ä", "ö", "Ö", "ü", "Ü", "ß", "ẞ",
        "æ", "Æ", "œ", "Œ", "ø", "Ø", "ł", "Ł",
        "đ", "Đ", "ð", "Ð", "þ", "Þ", "ħ", "Ħ",
        "ı", "ij", "IJ", "ŋ", "Ŋ", "ŧ", "Ŧ", "ſ", "ŀ", "Ŀ",
        "&", " "
      ],
      "ReplacementTargets": [
        "ae", "AE", "oe", "OE", "ue", "UE", "ss", "SS",
        "ae", "AE", "oe", "OE", "o", "O", "l", "L",
        "d", "D", "d", "D", "th", "TH", "h", "H",
        "i", "ij", "IJ", "n", "N", "t", "T", "s", "l", "L",
        "and", "-"
      ]
    }
  }
}

Choose configuration for the repository's content:

  • Hungarian: enable RemoveDiacritics, leave ApplyCustomReplacements disabled. With Lowercase also enabled, Árvíztűrő tükörfúrógép becomes arvizturo tukorfurogep. Spaces remain under the default invalid-character pattern; use the existing InvalidNameCharsPattern / ReplacementChar configuration if whitespace should become hyphens.
  • German: enable ApplyCustomReplacements. With only this step enabled, Fähre & Straße schön kühl becomes Faehre-and-Strasse-schoen-kuehl, and ÄÖÜẞ becomes AEOEUESS. RemoveDiacritics can also be enabled for remaining decomposable accents; custom replacement runs first, preserving the German ae/oe/ue results.
  • Mixed languages: the operator chooses the switches and customizes the list. There is no language detection or per-language rule selection. The prepared ö → oe and ü → ue pairs apply to all matching input: for example, custom replacement plus diacritic removal and lowercasing turns Hungarian Árvíztűrő tükörfúrógép into arvizturo-tuekoerfurogep.

RemoveDiacritics alone maps ä/ö/ü to a/o/u and leaves ß unchanged. The prepared list is a configurable starting point, not a guarantee of language-specific spelling. Stored DisplayName remains unchanged.

Configuration providers merge arrays by index. A shorter environment or JSON array does not clear remaining entries from the base configuration; customize the complete paired list in the base appsettings.json when removing prepared pairs. This is the existing configuration behavior.

The space-to-hyphen mapping does not change the repository's validation pattern. The existing InvalidNameCharsPattern and ReplacementChar behavior remains unchanged.

Runtime Settings override

The global /Root/System/Settings/ContentNaming.settings file can disable any of the four configured steps. For example:

{"Transliterate": false}

Effective state is host config && Settings value, with a missing Settings file/key allowing the configured behavior. Settings true cannot enable a feature disabled in host config. Removing an override restores configured behavior. Settings are read during name generation on the running repository and use the existing cache invalidation, without a new cache or endpoint. Before repository startup, the configured behavior applies.

Only the four feature switches have runtime overrides. Replacement arrays and the existing validation/replacement configuration remain startup settings. The override is global; the existing naming API has no content-context path.

Each veto disables its corresponding step. Effects can overlap: Transliterate itself removes many diacritics, so disabling only RemoveDiacritics does not preserve accents while transliteration remains enabled.

Compatibility and Unicode scope

  • With all switches absent or false, existing name generation is preserved and no Settings lookup is required.
  • Stored DisplayName, string input parameters, existing whitespace/validation logic, the encoding provider, and original extension handling remain unchanged. A retained original extension preserves its casing and characters.
  • Existing contents are not renamed when configuration or Settings changes; this is standard sensenet behavior.
  • AnyAscii handles Latin letters such as ß → ss, Æ → Ae, Œ → Oe, plus other scripts and symbols. It is context-free and may differ from language-specific spelling, introduce punctuation, or remove unsupported characters. Existing collision/empty-name handling applies.
  • AnyAscii's ISC notice is included in the NuGet package and copied into output/publish, including transitive host output. No provider registration or schema change is required. A package/image containing this change is needed; host config changes require restart, while Settings veto changes do not.

Validation and review

Verified this German-default configuration update with .NET SDK 10.0.403 (net8.0 tests):

  • 78/78 naming tests, including existing AutoNaming regressions, switch combinations, literal/order/deletion/invalid mapping behavior, real AnyAscii outputs, Unicode casing, original extensions and the unchanged encoding provider.
  • 24/24 targeted OData tests, including naming, stored DisplayName, runtime Settings and existing create/rename regressions.
  • 45/45 isolated configuration probes using the actual configuration reader and naming provider: each of the seven base configurations with all steps disabled, German custom-only (including uppercase pairs), German custom + diacritics + lowercase, German diacritics-only, Hungarian without custom, and mixed text with the German pairs (42 checks); plus no-host-config/no-built-in-preset behavior, unchanged standalone custom arrays, and the existing environment array-index merge behavior. These are separate .NET processes, not Docker-container runtime tests.
  • Scope/configuration review verifies identical 36-pair lists in all seven hosts and the documented JSON example, four false defaults, exactly the requested four added pairs in each host, unchanged C# and project files against both the preceding configuration commit and 0e4bb4e24319240e228b4c1984da07af480701b3, and a clean git diff --check.
  • The earlier independent feature-code review and packaging/transitive output/publish/license checks were completed on 0e4bb4e24319240e228b4c1984da07af480701b3. Those files are unchanged in this configuration-only follow-up; the packaging checks were not repeated.

Documentation: normalization and runtime overrides.

@VargaJoe VargaJoe changed the title Add opt-in normalization options to content naming Add opt-in normalization for generated content names Nov 2, 2026
@VargaJoe VargaJoe changed the title Add opt-in normalization for generated content names Add opt-in diacritic removal for generated content names Nov 2, 2026
@VargaJoe
VargaJoe requested a review from kavics November 2, 2026 21:11
@kavics

kavics commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Discussion needed.
It would be good to also transliterate the Latin-script characters (Aegean, Strasse—about 14 of them?), and to note in the documentation that full transliteration (for Chinese, Cyrillic, etc.) is too complex for us.

@VargaJoe VargaJoe changed the title Add opt-in diacritic removal for generated content names Add opt-in diacritic removal and lowercase content names Nov 3, 2026
@VargaJoe VargaJoe changed the title Add opt-in diacritic removal and lowercase content names Add configurable content name normalization with runtime Settings overrides Nov 3, 2026
@VargaJoe
VargaJoe added this pull request to the merge queue Nov 5, 2026
Merged via the queue into develop with commit d46875a Nov 5, 2026
7 checks passed
@VargaJoe
VargaJoe deleted the feature/content-naming-normalization branch November 5, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants