D20: find the partitions eight short AKAI images displaced

[bmxcode]

Eight of the 44 AKAI images are short of the disc they were made from. The container is faithful to the file — every MDX block decodes and emits exactly the bytes it should — and the file is not a complete copy of the disc: whole 32 KB blocks are missing, so everything after a gap sits that much nearer the front. ADR-0022 found it from inside a volume directory, ADR-0023 measured it across the collection and declined to act on it, and #25 recorded the cost — 109 of the 384 declared partitions have no header where the table puts them, and on these eight they are not absent, they are displaced.

This deliverable finds 39 of them.

Disc	Recovered	Volumes	Files	Samples	Written
AMG - Kickin' Lunatic Beats 2 CD1	7	121	7 723	7 308	7 293
AMG - Kickin' Lunatic Beats 2 CD2	7	100	6 203	5 912	5 912
AKAI.S3000.Sound.Library.6	7	84	1 054	955	955
Back In Time Records - Elektra Vox	5	33	661	463	463
Audio Factory - Classical Wild Takes	4	18	189	80	80
AKAI.S3000.Sound.Library.5	3	34	473	400	377
AKAI.S3000.Sound.Library.7	3	20	660	546	546
AMG - Global Trance Mission 2	3	22	217	144	139
Total	39	432	17 180	15 808	15 765

The 44 AKAI discs go from 275 partitions to 314, 2 154 volumes to 2 586, 68 997 files to 86 177. The collection goes from 89 156 samples to 104 921. Nothing moves on the other 36 discs, and partition 1 does not move on any disc at all.

The rule

A declared partition with no header at its declared position is searched for backwards, in the unit the container stores the disc in, for a header restating the size the table gave that partition — and never below the end of the partition already accepted. (ADR-0028)

Four constraints, each doing different work:

Backwards from the declared position, anchored to it. A short image has lost bytes; nothing moved away from the front. This is also what settles the index, which was the open question: sizes repeat within a disk — eight of Loop Soup's nine are 7680 blocks — so a header restating size N does not say which N it is, and 67 of a free scan's 72 first-pass hits were size-ambiguous. Because the search starts where the table puts partition k, what it finds is partition k or nothing. out/partition-N/ is the table's N on a displaced partition exactly as on a present one, and there is no case where it cannot be known.

In the unit the container states. SectorImage.granularity is new on the container contract: cooked bytes per unit this container stores the disc in — the thing an incomplete image is missing whole numbers of. Flat containers report one sector; MdxImage reports its measured block size scaled to the cooked stride, 32 768 here and 30 720 on a subchannel image. This is the ADR-0003 seam and it is answered in the direction that keeps both sides clean: a 32768 in fs/akai.py would be the container's business asserted by the layer that cannot know it, and an AKAI check in container/ would be the same leak the other way.

Confirmed by the size the table gave that partition — the same confirmation a declared position gets, plus the block count for this index.

Never below the end of the partition already accepted. This is the safety argument, and it is a bound on the search rather than a filter on its results, so an overlap cannot arise and cannot need resolving. Two things fall out rather than being added: partition 1 can never move, because its declared position is 0 and its window is empty; and accepted partitions never overlap in either direction, since a recovered partition ends before its declared end.

What answers ADR-0023, in its terms

ADR-0023 refused a signature search on three grounds. It was right on the evidence it had. One of those three grounds turns out to be a different fact than it looked.

"The signature is a sawtooth and audio reproduces it." Not audio. Digested, ProSamples vol.14's 153 matching blocks are its five real partition headers plus one 8192-byte block repeated 148 times, byte for byte; Global Trance Mission 2's 374 are three distinct blocks repeated 288, 58 and 19 times. Audio does not repeat 8 KB exactly 148 times. They are complete stale partition headers sitting in free blocks, with volume directories that parse — vol.14's is the pristine formatted state, VOLUME 001…VOLUME 100; Global Trance Mission 2's names real volumes, R+R KIT 1, REGGAE KIT 1. So the refusal stands and is firmer than it was: there is no byte test that separates these from a partition header, because they are partition headers, and no stricter signature would have helped. That is why the constraints above are about where a header may be found rather than what one looks like. The format doc's old reading is corrected rather than left standing beside the new text.

"Filtering matches by the file-yielding test lost real partitions." Not done, and Advanced Media Trax 3 — the disc that lost 94 volumes down to 72 — is now pinned in tests/test_discs.py at 9 declared, 9 present, 94 volumes, 2 938 files and nothing displaced, precisely so a future search that touches it fails.

"The gain comes from short images, where the audio is displaced with nothing reporting it." Answered by D19, and then measured: 15 765 of the 15 808 recovered samples, 99.7 %, carry a payload header whose id, valid flag and name agree with the entry that placed them, checked against a structure the search never consults. Five of the eight discs recover thousands of files with not one refusal between them.

The reason the payloads survive — which ADR-0023 could not have known — is that a gap removes whole blocks, so everything past one shifts by a constant. Inside a displaced partition the directory and its audio moved together and stay consistent. Displacement only breaks payload/directory agreement where a gap falls inside a partition read at its declared position, which is D19's 61 mismatches, all at the tail of one volume just past a gap.

The ProSamples discs are rejected by the rule, not by a list of names. Every one of their nearest matches is below the floor: seven discs match exactly one partition back — the previous partition's own header, seen through a size the disk repeats — and vol.12 and vol.14 match 70 and 21 blocks back, on stale copies in free space. Both kinds are inside a partition already read, and neither is examined.

What this deliberately does not do

70 declared partitions stay unread, and 60 of them are not absent. A header is there, inside a partition already being read: 31 would overlap one, 29 land exactly on one. The remaining 10 have no header at any position the search may look at.

Best Service - Alpha Dance I gains nothing at all. Its one displaced partition is four AKAI blocks — one container block, the same gap as Kickin' CD2 — inside partition 4. Truncating at the clash was rejected: a clash means one run of bytes that two partitions' bookkeeping both describe, and the audio can only belong to one of them; reading both would put the same blocks under two names, which is the failure Protozoa taught in ADR-0021. Relaxing the floor by one block to take it was rejected as a threshold with one specimen behind it. That disc's answer is no answer, and the ADR says so rather than tuning until it is not.

Re-placing the present partition instead — arguably the truer reading, since on a clash the present partition's declared extent is provably wrong — was rejected on the gate this deliverable is judged by: the 275 partitions read today are the whole established baseline, pinned per disc across four deliverables.

Issue #35, and whether this explains it

It does, and it is now measured rather than supposed. 103 of the 104 payloads refused across the collection are sitting intact, carrying their entry's own name, a whole number of container blocks earlier — one block back for Alpha Dance II's 21 and Library.1's 3, 134 for Library.3's one, one to four for the rest. The single exception is Loop Soup's known directory record that lands mid-sample. Same mechanism, one level down.

It is not fixed here, and the boundary is a real one rather than a scoping convenience: a partition has a declared position to anchor a search to and a file has only the chain its allocation map states. Placing a file's audio somewhere the map does not put it is the search ADR-0022 and ADR-0023 both refused, and it would need its own answer to which file is this? — a name comparison, on payloads whose name does not decode on 54 of 60.

The name check, for the third deliverable running

D19's name check — the guard against a displaced payload landing on a valid header of a different sample — still has zero unique positives. It now fires 104 times across roughly 72 000 samples and never once without the id and valid tests firing too.

This is worth saying plainly. D20 was the deliverable expected to give it one: it recovers 15 808 samples from exactly the kind of image the test was kept for, on discs whose audio is displaced by construction, and it produced no unique catch either. The test that motivated the work is again the one doing none of it. It stays, because it is the only test that asks whether a payload is this sample rather than a sample, and it is exercised synthetically alone — and that is now the third release in a row where its justification is an argument rather than a measurement.

Other findings, reported rather than buried

18 new stereo-join refusals, all on Library.6's recovered partition 5: -L/-R pairs whose halves declare different sample rates — PROPHE FX1 at 44 597 against 44 100. The joiner refuses to fuse a pair the disc disagrees with itself about and writes both mono halves, so nothing is lost. It is why the run's "skipped (damage)" line goes from 5 to 23 while no audio went missing.

Kickin' CD1 yields 7 723 files, which is exactly what ADR-0023's rejected signature scan measured for that disc by a different method — an independent check on the recovery that this branch did not perform.

Reporting

list prints the displacement under each partition heading and in the layout line — "11 partitions declared, 8 present in this image (7 of them displaced -- this image is short of the disc it was made from)" — and batch records that line per disc in the manifest as layout. Not per file: displacement is a property of the partition, and a per-file flag would repeat one fact 7 723 times.

Cost

list and batch are slower on a disc with missing partitions, because the search walks from a declared position back to the last accepted one. Over the whole collection the walk goes from 26 s to 45 s and a full batch from 48 s to 72 s. On a disc with nothing missing the cost is zero — the search never runs.

Verification

• ruff check, ruff format --check, pytest -q — green; uv tool install --editable . && samplerdisc --version — 0.3.0.
• SAMPLERDISC_TEST_DISCS=... pytest -q — green over all 79 images.
• New disc-backed pins: _AKAI_SHORT fixes per disc the declared/present counts, the displacement of every recovered partition in AKAI blocks, and the volumes, files, samples, written and refused counts for the displaced partitions alone. _AKAI_NOTHING_RECOVERED pins Alpha Dance I recovering nothing and the header being really there. _AKAI and _AKAI_FIRST_PARTITION gain Advanced Media Trax 3, ProSamples vol.14 and vol.54 as the falsifying cases.
• Every accepted payload's word count still agrees with its entry's declared size: 72 190 of 72 190. Every WAV still verifies against the disc it came from: 70 of 70 discs, 104 921 payloads, zero mismatches.
• README counts re-measured from a full batch run and a listing pass, not derived by arithmetic.

Closes #25.

🤖 Generated with Claude Code