Shape descriptors no live object carries are never pruned (35k of 43.7k on claude-code): retire from a full-trace carrier note

[proggeramlug]

Follow-up to #9706 / #9724. That PR cut the shape tables on the compiled claude-code TUI from 22.2 MB to 8.1 MB and the descriptor count from 68,661 to 43,724, and added census rows that say what the survivors are. The rows say most of them are dead weight:

shapes.descriptors                          entries=43724   bytes=3822856
shapes.descriptors.carried(live objects)    entries= 8664
shapes.descriptors.uncarried                entries=35060
shapes.descriptors.uncarried.cache_carrier  entries=    0
shapes.descriptors.uncarried.old_carrier    entries=    0
shapes.families.multi(families,extra)       entries=13141   bytes=21197
shapes.families.largest                     entries= 1832
shapes.ids_minted(process)                  entries=1053380

(PERRY_GC_CENSUS, third SIGUSR2 census at idle, after two full collections. "Carried" means some live shaped object is stamped with that ShapeId; the walk collects object_shape_stamp of every live GC_TYPE_OBJECT.)

35,060 of 43,724 descriptors are carried by no live object, are owned by no optimization cache (cache_carrier), and were not noted as carried by an old receiver in the last full trace (old_carrier). They are transition history that nothing can reach again, and nothing prunes them.

Why they survive

The only retirement path for a descriptor whose facts are no longer anyone's is prune_dead_shape_keys (crates/perry-runtime/src/object/shapes.rs), and it is keyed on the keys array dying: is_dead_owner(record.keys) || shape_keys_address_is_recycled(record.keys). A descriptor survives as long as its keys array does, whoever carries it. Two things keep keys arrays alive without carrying the descriptor:

1. The keyless family. keys == 0 and is_dead_owner(0) is false (gc/dead_owner.rs::attributed_owner_header rejects addr < GC_HEADER_SIZE), so a keyless descriptor is immortal by construction. Every keyless object that takes a per-object semantic generation — Object.create(proto) (prototype divergence, object/prototype_chain.rs → transition_object_shape_semantics), the first defineProperty/accessor install on it, or an attribute/accessor removal (object/descriptor_state.rs) — mints a descriptor (keys=0, count=0, live, generation=G) that is unique to that one object and stays after the object dies. shapes.families.largest = 1832 is almost certainly this family; it grows for the life of the process.
2. Shared keys arrays. A GC_FLAG_SHAPE_SHARED array is immutable and shared by every object that reached that transition, and it is also rooted by the transition cache (object/mod.rs TRANSITION_CACHE_GLOBAL, next_keys) and the shape inline cache (shape_cache_insert). So a per-object generation descriptor (K, n, live, G) minted for one object that has since died lives as long as K — i.e. as long as the shape is in use by anyone at all. Same for prefix-count versions (K, n<len) and live-bound variants nobody carries any more.

This was true before #9724 too (at ~330 B each instead of ~32 B + two 16-byte index slots), which is why the issue's "2.1× more descriptors than V8 maps" question is only half answered: V8 keeps transition-tree maps, but it does not keep one map per object that ever had a property redefined.

What a fix needs

A per-record carried-in-this-full-trace note, and a retirement pass gated on it:

• Set the note for every shaped receiver a full trace visits, not only non-nursery ones. Today gc/layout_slot_visit.rs::visit_gc_layout_slot_descriptors calls note_old_generation_carrier only when the receiver is outside the nursery (that is the feat(gc/shape): root and rewrite the keys edge from the ShapeId descriptor, not ObjectHeader.keys_array #8112 minor-GC rooting gate; young carriers emit the keys edge themselves and need no note). The new bit is a different question — "did anyone carry this during the last full trace" — so it wants its own flag (RECORD_FLAG_CARRIED_SEEN next to OLD_CARRIER_SEEN in shapes_store.rs), set unconditionally in that visit, and rotated by rotate_old_carrier_epoch_after_full_trace. A full trace enumerates everything reachable, including old-gen, gc_malloc'd large objects and the immortal globalThis bootstrap residents (gc/layout_tables.rs immortal window doc), so the note is complete exactly when the trace is.
• Retire, at the point prune_dead_shape_keys already runs after a synchronous full trace, every record that was not carried, is not cache_carrier, and is not named by any runtime table that can re-stamp it (next bullet). Never after a minor or a budgeted cycle: those do not enumerate every carrier.
• Every table that holds a ShapeId it may stamp onto an object is a carrier and must say so. The ones known today:
  • object/array_tail_transition.rs — already does (note_cache_carrier on insert, recompute_cache_carriers_after_full_trace after a full trace).
  • object/mod.rs transition cache (TRANSITION_CACHE_GLOBAL): each entry holds prev_shape_id and target_shape_id; a hit calls install_cached_object_shape_transition(obj, prev, target, next_keys), which stamps target without re-validating that the id still resolves in release builds. Retiring an uncarried target would put a receiver into the Tombstone deletes lose a live object's keys array under evacuating GC — Object.keys() returns empty, fields read NaN (main, gc-stress red) #9200 failure mode: stamped with an id that resolves to nothing, so Object.keys() is empty and every fixed-slot read is undefined. Either mark targets as carriers when the entry is inserted and recompute from table occupancy after a full trace (the array-tail cache's pattern), or make the hit path re-validate the id (shape_descriptor_by_id(target).is_some(), one slab probe).
  • object/mod.rs shape inline cache: ShapeCacheEntry::runtime_shape_id (birth stamp for compiled literals/classes, try_birth_stamp_preinstalled_shape). A retired id there only costs a fallback mint, but a mint means the id churns per birth; simpler to count the entry as a carrier.
  • Codegen module globals from js_object_shape_id_for_keys (the @perry_class_keys_* sibling): same fallback-to-mint story via try_birth_stamp_preinstalled_shape's shape_descriptor_by_id check. Counting them as carriers (they are process-global ids installed at module init) avoids the churn.
  • Per-thread: ids installed via install_external_shape_id on a worker are that worker's table entries; each agent decides for its own table, so nothing crosses threads.
  • Generated PIC sites compare a cached id against the receiver's stamp and miss on mismatch; a retired id in a PIC is harmless.
• Sabotage test in gc/tests/shape_keys_descriptor_edge.rs: build an object, give it a per-object generation (Object.setPrototypeOf or defineProperty through the runtime entry points), drop it, run a full collection, assert the descriptor is gone; then the inverse — keep the object alive, or park its id in the transition cache, run the same collection, assert the descriptor and the receiver's Object.keys() survive. The receiver-survives arm is the one that matters: a green "it was pruned" without it is Tombstone deletes lose a live object's keys array under evacuating GC — Object.keys() returns empty, fields read NaN (main, gc-stress red) #9200 again.

Expected size of the win

Modest in bytes, large in count: an uncarried descriptor now costs a 32-byte slab record plus roughly 24 bytes in by_facts and 8 in families, so 35 k of them are ~2.5–3.5 MB of the remaining 8.1 MB, and the keyless family stops growing without bound. The count would drop from 43.7 k towards the ~8.7 k carried plus whatever the caches hold, which is where the V8 comparison (39.7 k maps) becomes apples to apples.

How to measure

PERRY_GC_CENSUS=<path> on the compiled claude-code TUI, SIGUSR2 to the process at idle (the event loop turns it into a full collection and appends one JSON line); the shapes.descriptors.carried(live objects) / .uncarried rows are the before/after. #9724's PR description has the exact relink-and-census recipe; the baseline binaries from that run are under /root/worktrees/perry-9706/cc/ on perrymaster with their census lines in census/.

Refs: #9706, #9724, #8112 (carrier gates), #9200 (the retired-stamp failure mode), #8086 (where history retention started).