Virgin.root stored-data oddities require explicit migration policy (malformed nodes, placeholders, control-byte names, exotic types) #889

Rationale

Virgin.root contains stored-data oddities that the root-build migration must handle explicitly with loud policy decisions (preserve vs drop per item): malformed nodes, placeholder values, collision-prone names with control bytes, exotic types. These are not data corruption but genuine quirks that require migration strategy.

Current State

ODB Census findings (2026-08-11, preserved at planning/root_build_census_2026-08-11.txt):

1. Malformed root-level node:

   • NAME is a UserTalk assignment statement: user.tools.databases.[name].flInstalled = false
   • Type: ???? (undefined/empty)
   • sizeOf errors
   • Signature: editing accident stored as data
   • Decision needed: preserve or drop?

2. 106 ???? empty/undefined placeholder values:

   • Distributed throughout Virgin.root
   • Zero in guest DBs
   • Represent incomplete/broken table entries
   • Decision needed: preserve as-is or populate with defaults?

3. Value NAMES containing tabs and literal CR/LF bytes:

   • Real paths under Manila windowTypes
   • Real code under tcp.im aim driver
   • Example: names collide AFTER CR normalization (two distinct paths become identical)
   • Impact: file-per-value export needs collision-safe name escaping
   • pctcodec must cover control bytes (tabs, CR, LF)
   • Decision needed: escape on export or reject on migration?

4. Singleton exotic types confined to Virgin.root:

   • cRGB (2 instances)
   • qdrt (1 instance)
   • QDpt (1 instance)
   • exte (2 instances)
   • obj (1 instance)
   • tokn (1 instance)
   • Not in guest DBs; unclear if supported by v7 format
   • Decision needed: preserve or drop?

Desired Outcome

For each oddity category, establish explicit migration policy:

1. Document in ROOT_BUILD_FROM_SOURCE_PLAN.md Section 3.3
2. Decide: preserve (with justification) or drop (with rationale)
3. Implement migration logic (conditionals, type-specific handling)
4. Audit for data loss or unintended side effects

Recommendation

For each category:

1. Malformed node: DROP (editing accident, not valid data)

   • Log as "removed malformed node: [NAME]"
   • Verify no scripts reference it

2. ???? placeholders: PRESERVE (may be referenced by metadata)

   • OR: Survey references and drop unreferenced ones
   • Add note to schema: ???? is valid empty-value marker

3. Names with control bytes: ESCAPE on export (not on migration)

   • Use pctcodec: tabs → %09, CR → %0D, LF → %0A
   • File-per-value export collision-safe
   • In-database names unchanged (preserve character data)
   • Migration: no change (preserved as-is)

4. Exotic types: PRESERVE (legacy, may break if dropped)

   • Document in v7 schema: "legacy types from Frontier 8.x"
   • Test round-trip: read → write → read idempotent
   • Plan Phase 4: deprecation if not used

Exit Criteria

• Policy documented in ROOT_BUILD_FROM_SOURCE_PLAN.md Section 3.3
• Each oddity category has explicit decision (preserve/drop with rationale)
• Migration code implements decisions (conditional drops, type preservation)
• pctcodec updated to handle control bytes (tabs, CR, LF)
• Export tool tested with collision-prone names (escaping works)
• Audit: no data loss, no unintended side effects

Related Context

• Found by: ODB census 2026-08-11, preserved at planning/root_build_census_2026-08-11.txt
• Related issues: ROOT_BUILD_FROM_SOURCE_PLAN.md (migration strategy)
• Impact: Planned --export-root tool, root-build migration process
• Data scope: Virgin.root only (guest DBs clean)
• Phase context: Phase 2+ root-build work

Census Details

Specific examples:

• Malformed: user.tools.databases.[name].flInstalled = false (statement as node NAME)
• Placeholders: 106 values with type ????
• Control bytes: Manila windowTypes paths, tcp.im aim driver code
• Exotic types: cRGB x2, qdrt, QDpt, exte x2, obj, tokn

No precedent in guest DBs: All oddities are Virgin.root-only, suggesting they're legacy artifacts or editing accidents.

Note

These are not bugs or corruption but genuine data quirks that require explicit handling. The migration strategy must decide policy loudly (preserve with justification vs drop with rationale) to avoid silent data loss or unintended behavior.