blob: bail-safe blob reads via road-owned hands

[matthew-levan]

Blob reads on ml/bob went through u3_blob_mmap(): an anonymous mapping the caller unmapped by hand, recorded nowhere, so any u3m_bail or signal that longjmp'd while one was live leaked it. The exposed sites were the hottest: u3_blob_load, u3r_view_padd, paired views in cat/aor/shax/argon2, jam across _cs_jam_fib_grow, every loop that allocates under a view. Each mapping also charged the file size against process VA.

Road-owned hands (pkg/noun/blob.c)

A u3_blob_hand is an open blob file owned by the road that opened it: fd, file size, cached bit-length, live-view count, optional read-only mapping. Nodes link by pointer.

• Inner roads list hands from bob_p, a former fut_w slot in u3a_road (layout and offset asserts unchanged), allocate them in their own heap, deduplicate by blob id, and retain them until the road falls. A trap that reads one blob a thousand times opens it once. A nested road borrows an inner ancestor's hand without listing it.
• Home roads never fall and every caller there is C with an explicit close, so their hands live on the C heap under a static head, one per open, freed on close. Nothing about hands is in the snapshot or needs marking.
• u3_blob_data / u3_blob_data_wid: the file mapped read-only, owned by the hand. Bytes past EOF read as zero through the last page (u3_blob_hand_pad); a wider request reserves anonymous zero pages and fixes the file's pages over their head, so one pointer covers any width. A remap is refused while another live view holds the hand; a file that shrank under the hand is refused rather than mapped. Windows maps exactly the file and has no fixed mappings, so it falls back past the file's pages.
• u3_blob_drain(rod) / u3_blob_drain_kids(): close every hand held by a road, or by every road below home.
• Past 256 retained hands, or on EMFILE, an inner road evicts its oldest idle hand and retries; with none idle it bails %file, so a resource condition never reads as a deterministic nock failure. The home road returns 0. Replay maps %file to a save-and-exit.

Unwinding

• u3m_fall drains the road it leaves. Every road exits there except by a signal caught at the top, so this replaces the sweep in u3m_bail and the leak check in u3m_soft_top.
• _cm_signal_reset drains the kid chain before cutting it. All signal-driven unwinds (SIGINT, SIGTERM, SIGVTALRM, stack overflow, loom %meme) land there through the single catch in u3m_soft_top.
• Every fd and mapping mutation runs inside u3m_crit_enter / u3m_crit_leave: an fd is on a list before a signal can unwind the opener, and a drain finishes unlinking before recovery can drain again. POSIX holds SIGINT/SIGTERM/SIGVTALRM with sigprocmask; the Windows rsignal emulation records a held signal as pending and delivers it at unblock.

u3r_view is the only way to read a bob

u3r_view is a plain struct, {byt_y, len_w, kin_e, han_u, raw_d}, with two modes over one type:

• Windowed: u3r_view_open opens the hand and maps nothing; u3r_view_read copies any byte range, from the hand's mapping if the road already has one and by pread otherwise, zero-fills past the atom, and returns the count that came from the atom. u3r_view_met gives the full-width bit length. The fixed-width accessors, u3r_sing/u3r_comp, u3r_blob_load/u3r_blob_cut, u3r_met_d (which replaces u3r_blob_met and the bob dispatch in met and both jam encoders), and the king's http and unix streaming all read this way. rip reads its atom as a word buffer through a flat view and no longer materializes a bob.
• Flat: u3r_view_init and u3r_view_padd alias the hand's mapping, widened to the pad, so a bob pad never allocates. A direct atom's value sits in raw_d; a pad over a loom atom is a loom pad.

A view owns nothing. It borrows the loom atom, the hand's mapping, or raw_d, and u3r_view_done releases its hold. u3r_view_open is the sole opener: every call on a hand sits inside a u3r_view_* function, and outside retrieve.c only blob.c and the tests name one. retrieve.h states the lifetime rule, the semantics of each mode, and the memory behavior per kind. u3_blob_mmap/u3_blob_umap and u3r_blob_mmap/u3r_blob_umap are removed.

Mars is the only writer

u3_blob_move_stg is the one function that creates bob/<mug>/<seq>. u3_blob_stage and u3_blob_stage_fd write bytes to .urb/bob/stg/; u3_blob_save and u3_blob_save_fd are stage-then-install for mars and the tests. The king never calls them: http bodies at or past the threshold and reassembled mesa packets are staged, sent in a %blob writ, and their %request or %heer is planned from the install callback, so every king-held bob is leased from the moment it exists. The http request is named by connection and sequence across the wait, since the client may close it first; a refused install answers 500, and a refused packet is dropped for the sender to retransmit. The unix initial scan runs before mars can install anything and reads a large file into the loom; the watcher stages later changes as before.

King side

u3_hbod blob chunks are {vue_u, off_d, len_w} sharing one heap-allocated windowed view; _http_hgen_send reads each chunk into a heap buffer only as h2o asks for it, so a streamed response holds at most two U3_HTTP_BOB_CHUNK buffers regardless of blob size. The tail chunk owns the view. A bob whose blob cannot be opened fails the response with a 500 instead of serving zeros. _unix_write_file_hard streams through a view.

Bank marking

u3v_mark marked the blob bank HAMT but not the walloc'd u3a_blob records it points at, so u3a_sweep (|mass, gc, pack) freed every record as a leak and every bob then named a garbage file. The records are marked and reported under "blob bank / records".

|mass

The king logs king: blob handles: N on %mass; the serf prints blob handles: N after its report (0 between events). Scry-served files never give the king a bob: the range is folded into the scry, arvo slices with cut, and each response is a plain atom.

Tests (blob-test, 54)

Hand lifetime; no sharing on the home road; one hand across a thousand open/close pairs on an inner road, released at fall; a grandchild borrowing an inner parent's hand; the retention cap; EMFILE on the home road and eviction then %file on an inner road; drains on bail, %meme under a view, cue bailing under a view, u3m_signal, a real SIGVTALRM while the road's list is churned, a real SIGINT outside and inside a critical section; leaked normal return; home hands surviving inner unwinds; a lone view widening its mapping in place and a second live view forcing the fallback pad; flat pads inside the page, 100000 bytes past it, loom, direct, truncated; windowed views over a bob, a loom atom, and a direct atom, read inside and past the end, never mapping; readers under a live mapping; wipe under a live hand; u3_blob_stop; truncation under an open hand; every entry point against a missing file; window-boundary lengths for met and compare; empty files; staging from a buffer and a descriptor installing to the same blob as a direct save, with a short source refused; 300 concurrent hands; both jam encoders on a multi-window bob byte-identical to the materialized atom; rsh/end/cut against the materialized atom; mark+sweep keeping the bank record under u3o_leak_crash. POSIX-only tests compile out on Windows.

Follow-ups

• u3_blob_save_fd and u3_blob_move_stg still mmap the staging file to mug and dedup it; streaming them needs an incremental mug.
• u3_blob_bsink's staging fd is not registered; both callers run on the home road.
• Incremental-capable hash jets still take a flat view.
• The Windows stack-overflow job can fail when ASLR places a loader section inside the loom's fixed window before u3_disk_stake runs; a fallback base belongs to the Windows loom branch.