From 7fa1fd2dfa78ddd8779fb1f34367b61aaf2bc2bd Mon Sep 17 00:00:00 2001 From: Peter Tos <38345014+bmxcode@users.noreply.github.com> Date: Fri, 21 Aug 2026 18:37:54 +1000 Subject: [PATCH] Docs: re-measure the collection, and correct a partition count by one The README's "Tested against" block still described the run before D15 read every AKAI partition, and disagreed with docs/README.md on the same page. Everything in it is re-measured over all 79 images. discs converted 69 of 79 -> 72 of 79 samples 47742 -> 89125 stereo pairs 4422 -> 15796 damage skips 20 -> 97 time 31s -> 46s distinct rates 908 -> 1047 (6000-49999, was 6000-48000) silent whole-file 10 -> 275 The byte-identity claim was quoted twice on one page at two different sizes -- 22 320 payloads in one paragraph and 40 244 in another. It is now one statement: 70 of 70 discs match exactly, 89 125 payloads, zero mismatches, comparing multisets of SHA-256 over the PCM per disc. The two audio CDs are excluded and said to be excluded, because a track is cut from a stream by a cue and is not a run of bytes anywhere on the disc. The silence count moving from 10 to 275 is not a regression. Every one of the 275 matches the disc exactly, so it is what the discs hold: 267 are the blank 15G-KIT...Z slots on ProSamples vol.15, six are the Proteus "Dead Air" samples the page already described, two are on a Roland disc. D15 unlocked 3.4x more AKAI content and the blank kit slots came with it. Also corrected: docs/formats/akai-fs.md and three comments in fs/akai.py said the 44 AKAI discs hold 276 partitions. The table declares 384 and 275 are present in the images. Measured twice by separate routes, and 275 is what len(list(partitions(...))) sums to -- the same expression test_akai_discs_list_their_volumes_and_files already pins per disc for nine of them, which is what makes this a correction rather than a preference. The figures beside it were re-checked and are exact: 2 154 volumes, 68 997 files. The earlier 276 could not be reproduced. ADR-0023 also says 276 and is left alone, as ADR-0019 was left alone through the disc renames: an ADR records what was measured when the decision was made. The correction is in the changelog instead. No source behaviour changes. The only non-doc edits are three comments. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 4 ++++ README.md | 45 ++++++++++++++++++-------------------- docs/README.md | 2 +- docs/formats/akai-fs.md | 6 ++--- src/samplerdisc/fs/akai.py | 6 ++--- 5 files changed, 32 insertions(+), 31 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 30fd66c..227a642 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -66,6 +66,10 @@ Notable changes to `samplerdisc`. Format-level findings live in [docs/formats/]( - **A clean disc reported itself as a damaged one.** `Skipped` covered both an entry lost to damage and an entry deliberately not written, and the summary called every one of them damage — `vol.42` printed *"skipped 423 damaged or unreadable entries"* when all 423 were sounds already written under another name. Duplicates are now counted and reported apart, in the summary and in the batch manifest. - **`--keep-originals` no longer calls its output "AKAI files"** in the summary. An ISO 9660 disc keeps EXS24 and HALion instruments through the same path. +- **The README's "Tested against" figures were two deliverables out of date, and disagreed with `docs/README.md`.** They still described the run before D15 read every AKAI partition — 69 of 79 discs and 47 742 samples against the 72 of 79 and 89 125 the collection now yields — and the byte-identity check was quoted at 22 320 and 40 244 payloads in two different places on the same page. Both are re-measured over all 79 images, along with the rate spread (1 047 distinct values, 6 000–49 999 Hz, where the page said 908 and 6 000–48 000) and the whole-file silence count (275, where it said ten). + +- **The partition total quoted across the docs was one too many.** `docs/formats/akai-fs.md` and three comments in `fs/akai.py` said the 44 AKAI discs hold **276** partitions. Measured twice by separate routes, the table declares 384 and **275** are present in the images — and 275 is what `len(list(partitions(...)))` sums to, the same expression `test_akai_discs_list_their_volumes_and_files` already pins per disc for nine of them. The figures beside it are unaffected and were re-checked: 2 154 volumes and 68 997 files, both exact. The earlier 276 could not be reproduced. [ADR-0023](docs/adr/0023-partitions-come-from-the-table-the-disc-declares.md) keeps its original wording, being a historical record. + ### Known limits - **The ISO 9660 directory hierarchy is not preserved.** A disc's audio is written flat into one directory per volume, so `PS-34 AIFF …/056_Ballad de Boo/34a-bas-56Dmin.aif` becomes `34a-bas-56Dmin.wav` and the folder that grouped it by tempo is gone. Nothing is lost on these discs — Best Service named every file uniquely, and all thirteen list zero collisions after flattening — but a disc that reused a basename would rely on `unique_path` suffixing it. diff --git a/README.md b/README.md index 7acba29..2212a6f 100644 --- a/README.md +++ b/README.md @@ -71,43 +71,40 @@ Compressed `.mdx` is the piece no other open-source tool reads today. The format ## Tested against -79 disc images from two archive.org collections — [retro-sample-cds](https://archive.org/details/retro-sample-cds) and [archive-oldschoolscds](https://archive.org/details/archive-oldschoolscds). Fifty-two flat `.iso`/`.bin`, seventeen compressed `.mdx`, five raw CD images, two `.nrg`, one `.mds`/`.mdf` pair, two audio CDs. +79 disc images from three archive.org collections — [retro-sample-cds](https://archive.org/details/retro-sample-cds), [archive-oldschoolscds](https://archive.org/details/archive-oldschoolscds) and [Best Service ProSamples](https://archive.org/details/best-service-pro-samples-vol.-12-dance-vocals-akai-1-cd). Fifty-two flat `.iso`/`.bin`, seventeen compressed `.mdx`, five raw CD images, two `.nrg`, one `.mds`/`.mdf` pair, two audio CDs. | | | |---|---| -| Discs converted | 69 of 79 | -| Samples | 47 742 | -| Stereo pairs rejoined | 4 422 | +| Discs converted | 72 of 79 | +| Samples | 89 125 | +| Stereo pairs rejoined | 15 796 | | Audio CD tracks | 161 | -| Entries skipped (damage) | 20 | -| Time | 31 s | +| Duplicate audio suppressed | 5 719 | +| Entries skipped (damage) | 97 | +| Time | 46 s | -Every extracted WAV was checked against the bytes on the disc it came from — **all 22 320 are byte-identical**, none unreadable, none zero-length. Ten are silent for their whole length and are meant to be: they are named `Dead Air`, on a Proteus library that ships silence as a sample. +By filesystem: -Sample rates run from 6 000 to 48 000 Hz across 908 distinct values. The odd ones are real — E-mu writes rates like 24 444 and 27 778, and AKAI uses 33 075 (¾ of 44 100) and 29 400 (⅔) to trade bandwidth for memory. They are carried through exactly as the disc states them and never rounded. +| | Discs | Samples | Stereo pairs | Skipped | +|---|---:|---:|---:|---:| +| AKAI | 44 | 56 394 | 14 449 | 97 | +| E-mu `EMU3` | 7 | 14 738 | 6 | 0 | +| ISO 9660 | 15 | 11 601 | — | 0 | +| Roland `S770 MR25A` | 5 | 6 392 | 1 341 | 0 | -The ten that do not convert are accounted for: one S-550 disc present as both `.iso` and `.nrg` — a different format from the S-7xx and not yet read ([ADR-0014](docs/adr/0014-one-backend-per-on-disc-format.md)) — two Digidesign SampleCell discs, one audio CD with no cue sheet present as both `.mdx` and `.cdr`, three Emulator IV discs that list their banks but do not extract them, and one ISO 9660 disc holding E-mu `.EBL` banks rather than audio. +Every WAV was checked against the disc it came from — **70 of 70 discs match exactly**, comparing multisets of SHA-256 over the PCM per disc rather than going via filenames, so duplicate names cannot mask a mismatch and no path is guessed. 89 125 payloads, zero mismatches. The two audio CDs are not in that count: their tracks are cut from a stream by a cue, so there is no run of bytes on the disc to compare a track against. -Every WAV was checked against the disc it came from — **67 of 67 discs match exactly**, comparing multisets of SHA-256 over the PCM per disc rather than going via filenames, so duplicate names cannot mask a mismatch and no path is guessed. 40 244 payloads, zero mismatches. +105 082 WAV files were written in all — the samples, the stereo joins and the audio CD tracks. None is unreadable and none is zero-length. **275 are silent for their whole length, and every one of them matches the disc exactly**: 267 are the blank `15G-KIT…Z` slots on `ProSamples vol.15`, six are on a Proteus library that ships `Dead Air` as a sample, and two are on a Roland disc. That is what the discs hold, not something the decoder did. -The five Roland S-7xx discs contribute **6 392 samples and 1 341 stereo pairs, with nothing skipped**. +Sample rates run from 6 000 to 49 999 Hz across 1 047 distinct values. The odd ones are real — E-mu writes rates like 24 444 and 27 778, and AKAI uses 33 075 (¾ of 44 100) and 29 400 (⅔) to trade bandwidth for memory. They are carried through exactly as the disc states them and never rounded. -### Best Service ProSamples +The 97 damage skips are almost all one thing: 92 payloads that do not begin with an AKAI sample header, concentrated on seven `.mdx` images. The rest are four implausible sample rates and one stereo pair whose halves declare 44 033 and 44 100, so the joiner refuses to fuse them and writes both mono halves instead. -A third collection, [29 discs](https://archive.org/details/best-service-pro-samples-vol.-12-dance-vocals-akai-1-cd), added after the two above and measured separately. Sixteen are AKAI and thirteen are ISO 9660 discs holding plain audio. +The seven that do not convert are accounted for: one S-550 disc present as both `.iso` and `.nrg` — a different format from the S-7xx and not yet read ([ADR-0014](docs/adr/0014-one-backend-per-on-disc-format.md)) — two Digidesign SampleCell discs, one audio CD with no cue sheet present as both `.mdx` and `.cdr`, and one ISO 9660 disc holding E-mu `.EBL` banks rather than audio. -| | | -|---|---| -| Discs converted | 29 of 29 | -| Samples | 21 435 | -| Stereo pairs rejoined | 4 930 | -| WAV files written | 26 365 | -| Duplicate AIFF suppressed | 5 719 | -| Entries skipped (damage) | 0 | - -Every one of the 26 365 files written back reads as a valid WAV with audio in it. The single entry not written that was not a duplicate is a stereo pair on `vol.20` whose two halves declare different rates — 44 033 against 44 100 — so the joiner refuses to fuse them and both mono halves are written instead. +### The AIFF twins -The thirteen ISO 9660 discs ship each sound twice, once as AIFF and once as WAV. 6 033 of the 7 498 AIFF hold audio that is already coming out as a WAV, and those are written once; the other 1 779 are converted, along with 314 whose AIFF carries a root key or a loop that its WAV twin does not. `vol.43` is the reason the check is on the audio and not the filename: its 1 386 AIFF all share a name with a WAV and **none of them share its audio**, being mastered a few frames longer ([ADR-0024](docs/adr/0024-the-aiff-twin-is-converted-and-deduplicated.md)). +The thirteen ISO 9660 ProSamples discs ship each sound twice, once as AIFF and once as WAV. 6 033 of the 7 498 AIFF hold audio that is already coming out as a WAV, and those are written once; the other 1 465 are converted, along with 314 whose AIFF carries a root key or a loop that its WAV twin does not. `vol.43` is the reason the check is on the audio and not the filename: its 1 386 AIFF all share a name with a WAV and **none of them share its audio**, being mastered a few frames longer ([ADR-0024](docs/adr/0024-the-aiff-twin-is-converted-and-deduplicated.md)). These discs are also the only place in this project where the correct output is known independently. Where a sound exists as both, the publisher's own WAV says what the AIFF conversion should produce — including the loop convention the AIFF spec leaves open, settled on 195 pairs out of 195. diff --git a/docs/README.md b/docs/README.md index 8318f4f..abaa941 100644 --- a/docs/README.md +++ b/docs/README.md @@ -57,7 +57,7 @@ All of that vanishes into a working parser. Six months on, the code says *what* | D15 | Every partition of an AKAI disc, from the table it declares ([ADR-0023](adr/0023-partitions-come-from-the-table-the-disc-declares.md)) | done | | D16 | AIFF payloads converted, deduplicated against their WAV twin, and EXS24/HALion instruments kept ([ADR-0024](adr/0024-the-aiff-twin-is-converted-and-deduplicated.md)) | done | -Across the local collection, by listing: 71 of 79 images claimed, 2 578 volumes, 110 989 files, 77 620 of them samples. The AKAI discs are 44 of those images and 68 997 of those files, read across 276 partitions — before D15 they were 14 670 files, because only the partition at the origin was read. +Across the local collection, by listing: 71 of 79 images claimed, 2 578 volumes, 110 989 files, 77 620 of them samples. The AKAI discs are 44 of those images and 68 997 of those files, read across 275 partitions — before D15 they were 14 670 files, because only the partition at the origin was read. ## What is not done diff --git a/docs/formats/akai-fs.md b/docs/formats/akai-fs.md index 1b2f423..74a6744 100644 --- a/docs/formats/akai-fs.md +++ b/docs/formats/akai-fs.md @@ -30,7 +30,7 @@ Getting index 10 wrong is the classic failure and it is not obvious: `KICKIN B0- | `0x70A` | **block allocation map** — one u16 per block, as many as `0x00` declares | | `0x4500` | **partition table** — the disk's own list of its partitions, in the first partition only | -`0x02`, `0xC6` and `0xC8` all hold on all **276 partitions** of the 44 AKAI discs, and together they are what identifies a partition header. The two size fields matter most: a block count with no echo behind it is not a partition's, which is a firmer test than anything about the image's length. +`0x02`, `0xC6` and `0xC8` all hold on all **275 partitions** of the 44 AKAI discs, and together they are what identifies a partition header. The two size fields matter most: a block count with no echo behind it is not a partition's, which is a firmer test than anything about the image's length. **The pattern at `0x02` is a rising sawtooth, and sample data reproduces it.** As 16-bit PCM, `3333 × i` is a saw wave, so audio does match it — 374 blocks of one disc's free space carry a complete header prefix, every one in a block the allocation map calls free. That is why a partition header is *confirmed* where the table says one is, and never scanned for ([ADR-0023](../adr/0023-partitions-come-from-the-table-the-disc-declares.md)). @@ -76,7 +76,7 @@ At `0x70A`, immediately after the volume directory's hundred slots: **one u16 pe A file's extent is the chain from its start block to `0xC000`. **That is what verifies the map rather than merely making it plausible**: the chain length and the file size are stated by two different structures, and across all 44 AKAI discs they agree for **14 607 of 14 607 files** in the first partitions — exactly, with no disc disagreeing anywhere. -Read across all 276 partitions the figure is **68 267 of 68 284**, and the seventeen exceptions are one thing rather than a scatter: every one is a `MULTI FILE` — type `m`, all on `AKAI.S3000.Sound.Library.1` — whose chain runs exactly one block past what its size needs. A multi appears to be allocated a spare block. It is the only kind that disagrees anywhere. +Read across all 275 partitions the figure is **68 267 of 68 284**, and the seventeen exceptions are one thing rather than a scatter: every one is a `MULTI FILE` — type `m`, all on `AKAI.S3000.Sound.Library.1` — whose chain runs exactly one block past what its size needs. A multi appears to be allocated a spare block. It is the only kind that disagrees anywhere. The exclusion behind that figure is itself the finding. Five volumes sit on blocks the map calls free, and their files' chains are gone with them; four of those five hold 63 files that read perfectly. These are **deleted volumes**: the blocks went back to the free list, and because the medium is a mastered CD-ROM nothing ever reused them, so the directory and the audio are still there to be read. They are listed with their files like any other volume. @@ -119,7 +119,7 @@ A partition's own blocks are numbered from **its** start, so the same block numb Where the image holds no header at a position the table declares, it is **skipped and not searched for**. On the discs where that happens the header turns up displaced by a whole number of the container's 32 KB blocks — the image is short of the disc it was made from, which is the fault below and not a filesystem to go hunting through. -Across the 44 discs the table places 276 partitions holding **2 154 volumes and 68 997 files**, against the 448 volumes and 14 670 files of the first partitions alone. +Across the 44 discs the table declares 384 partitions and 275 are present in the images, holding **2 154 volumes and 68 997 files**, against the 448 volumes and 14 670 files of the first partitions alone. ## An image can be short diff --git a/src/samplerdisc/fs/akai.py b/src/samplerdisc/fs/akai.py index 0fb73e7..30afebc 100644 --- a/src/samplerdisc/fs/akai.py +++ b/src/samplerdisc/fs/akai.py @@ -118,7 +118,7 @@ #: See docs/formats/akai-fs.md and ADR-0023. PARTITION_TABLE_OFFSET = 0x4500 -#: 196 bytes of constant at 0x02, byte-identical across all 276 partitions of +#: 196 bytes of constant at 0x02, byte-identical across all 275 partitions of #: the 44 discs: 3333 x i as u16 LE, i = 0..97, wrapping. Nothing is known to #: read it and nothing on any disc varies with it, but a partition header #: carries it, which is what confirms a partition is where the table says. @@ -130,7 +130,7 @@ HEADER_PATTERN = struct.pack("<98H", *(3333 * step & 0xFFFF for step in range(98))) #: The header restates its own size. The u16 at 0xC6 is the block count at 0x00 -#: plus this bias, and the u16 at 0xC8 is 47; both hold on all 276 partitions +#: plus this bias, and the u16 at 0xC8 is 47; both hold on all 275 partitions #: measured. Two fields of the header agreeing is what lets a candidate be #: confirmed by the disc rather than by the arithmetic that placed it. SIZE_ECHO_OFFSET = 0xC6 @@ -269,7 +269,7 @@ def allocation_map(image: SectorImage, offset: int) -> list[int]: from one, instead of a map invented out of whatever follows the header. What makes a count sane is the header restating it: the u16 at 0xC6 is the - block count plus a fixed bias on all 276 partitions measured, so a count + block count plus a fixed bias on all 275 partitions measured, so a count with no echo behind it is not a partition's and gets no map. That is a firmer test than the image's length, which was the previous one and refused the map to a partition the image merely *ends inside* -- leaving two volumes