Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,10 @@ Notable changes to `samplerdisc`. Format-level findings live in [docs/formats/](
- **A clean disc reported itself as a damaged one.** `Skipped` covered both an entry lost to damage and an entry deliberately not written, and the summary called every one of them damage — `vol.42` printed *"skipped 423 damaged or unreadable entries"* when all 423 were sounds already written under another name. Duplicates are now counted and reported apart, in the summary and in the batch manifest.
- **`--keep-originals` no longer calls its output "AKAI files"** in the summary. An ISO 9660 disc keeps EXS24 and HALion instruments through the same path.

- **The README's "Tested against" figures were two deliverables out of date, and disagreed with `docs/README.md`.** They still described the run before D15 read every AKAI partition — 69 of 79 discs and 47 742 samples against the 72 of 79 and 89 125 the collection now yields — and the byte-identity check was quoted at 22 320 and 40 244 payloads in two different places on the same page. Both are re-measured over all 79 images, along with the rate spread (1 047 distinct values, 6 000–49 999 Hz, where the page said 908 and 6 000–48 000) and the whole-file silence count (275, where it said ten).

- **The partition total quoted across the docs was one too many.** `docs/formats/akai-fs.md` and three comments in `fs/akai.py` said the 44 AKAI discs hold **276** partitions. Measured twice by separate routes, the table declares 384 and **275** are present in the images — and 275 is what `len(list(partitions(...)))` sums to, the same expression `test_akai_discs_list_their_volumes_and_files` already pins per disc for nine of them. The figures beside it are unaffected and were re-checked: 2 154 volumes and 68 997 files, both exact. The earlier 276 could not be reproduced. [ADR-0023](docs/adr/0023-partitions-come-from-the-table-the-disc-declares.md) keeps its original wording, being a historical record.

### Known limits

- **The ISO 9660 directory hierarchy is not preserved.** A disc's audio is written flat into one directory per volume, so `PS-34 AIFF …/056_Ballad de Boo/34a-bas-56Dmin.aif` becomes `34a-bas-56Dmin.wav` and the folder that grouped it by tempo is gone. Nothing is lost on these discs — Best Service named every file uniquely, and all thirteen list zero collisions after flattening — but a disc that reused a basename would rely on `unique_path` suffixing it.
Expand Down
45 changes: 21 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,43 +71,40 @@ Compressed `.mdx` is the piece no other open-source tool reads today. The format

## Tested against

79 disc images from two archive.org collections — [retro-sample-cds](https://archive.org/details/retro-sample-cds) and [archive-oldschoolscds](https://archive.org/details/archive-oldschoolscds). Fifty-two flat `.iso`/`.bin`, seventeen compressed `.mdx`, five raw CD images, two `.nrg`, one `.mds`/`.mdf` pair, two audio CDs.
79 disc images from three archive.org collections — [retro-sample-cds](https://archive.org/details/retro-sample-cds), [archive-oldschoolscds](https://archive.org/details/archive-oldschoolscds) and [Best Service ProSamples](https://archive.org/details/best-service-pro-samples-vol.-12-dance-vocals-akai-1-cd). Fifty-two flat `.iso`/`.bin`, seventeen compressed `.mdx`, five raw CD images, two `.nrg`, one `.mds`/`.mdf` pair, two audio CDs.

| | |
|---|---|
| Discs converted | 69 of 79 |
| Samples | 47 742 |
| Stereo pairs rejoined | 4 422 |
| Discs converted | 72 of 79 |
| Samples | 89 125 |
| Stereo pairs rejoined | 15 796 |
| Audio CD tracks | 161 |
| Entries skipped (damage) | 20 |
| Time | 31 s |
| Duplicate audio suppressed | 5 719 |
| Entries skipped (damage) | 97 |
| Time | 46 s |

Every extracted WAV was checked against the bytes on the disc it came from — **all 22 320 are byte-identical**, none unreadable, none zero-length. Ten are silent for their whole length and are meant to be: they are named `Dead Air`, on a Proteus library that ships silence as a sample.
By filesystem:

Sample rates run from 6 000 to 48 000 Hz across 908 distinct values. The odd ones are real — E-mu writes rates like 24 444 and 27 778, and AKAI uses 33 075 (¾ of 44 100) and 29 400 (⅔) to trade bandwidth for memory. They are carried through exactly as the disc states them and never rounded.
| | Discs | Samples | Stereo pairs | Skipped |
|---|---:|---:|---:|---:|
| AKAI | 44 | 56 394 | 14 449 | 97 |
| E-mu `EMU3` | 7 | 14 738 | 6 | 0 |
| ISO 9660 | 15 | 11 601 | — | 0 |
| Roland `S770 MR25A` | 5 | 6 392 | 1 341 | 0 |

The ten that do not convert are accounted for: one S-550 disc present as both `.iso` and `.nrg` — a different format from the S-7xx and not yet read ([ADR-0014](docs/adr/0014-one-backend-per-on-disc-format.md)) — two Digidesign SampleCell discs, one audio CD with no cue sheet present as both `.mdx` and `.cdr`, three Emulator IV discs that list their banks but do not extract them, and one ISO 9660 disc holding E-mu `.EBL` banks rather than audio.
Every WAV was checked against the disc it came from — **70 of 70 discs match exactly**, comparing multisets of SHA-256 over the PCM per disc rather than going via filenames, so duplicate names cannot mask a mismatch and no path is guessed. 89 125 payloads, zero mismatches. The two audio CDs are not in that count: their tracks are cut from a stream by a cue, so there is no run of bytes on the disc to compare a track against.

Every WAV was checked against the disc it came from — **67 of 67 discs match exactly**, comparing multisets of SHA-256 over the PCM per disc rather than going via filenames, so duplicate names cannot mask a mismatch and no path is guessed. 40 244 payloads, zero mismatches.
105 082 WAV files were written in all — the samples, the stereo joins and the audio CD tracks. None is unreadable and none is zero-length. **275 are silent for their whole length, and every one of them matches the disc exactly**: 267 are the blank `15G-KIT…Z` slots on `ProSamples vol.15`, six are on a Proteus library that ships `Dead Air` as a sample, and two are on a Roland disc. That is what the discs hold, not something the decoder did.

The five Roland S-7xx discs contribute **6 392 samples and 1 341 stereo pairs, with nothing skipped**.
Sample rates run from 6 000 to 49 999 Hz across 1 047 distinct values. The odd ones are real — E-mu writes rates like 24 444 and 27 778, and AKAI uses 33 075 (¾ of 44 100) and 29 400 (⅔) to trade bandwidth for memory. They are carried through exactly as the disc states them and never rounded.

### Best Service ProSamples
The 97 damage skips are almost all one thing: 92 payloads that do not begin with an AKAI sample header, concentrated on seven `.mdx` images. The rest are four implausible sample rates and one stereo pair whose halves declare 44 033 and 44 100, so the joiner refuses to fuse them and writes both mono halves instead.

A third collection, [29 discs](https://archive.org/details/best-service-pro-samples-vol.-12-dance-vocals-akai-1-cd), added after the two above and measured separately. Sixteen are AKAI and thirteen are ISO 9660 discs holding plain audio.
The seven that do not convert are accounted for: one S-550 disc present as both `.iso` and `.nrg` — a different format from the S-7xx and not yet read ([ADR-0014](docs/adr/0014-one-backend-per-on-disc-format.md)) — two Digidesign SampleCell discs, one audio CD with no cue sheet present as both `.mdx` and `.cdr`, and one ISO 9660 disc holding E-mu `.EBL` banks rather than audio.

| | |
|---|---|
| Discs converted | 29 of 29 |
| Samples | 21 435 |
| Stereo pairs rejoined | 4 930 |
| WAV files written | 26 365 |
| Duplicate AIFF suppressed | 5 719 |
| Entries skipped (damage) | 0 |

Every one of the 26 365 files written back reads as a valid WAV with audio in it. The single entry not written that was not a duplicate is a stereo pair on `vol.20` whose two halves declare different rates — 44 033 against 44 100 — so the joiner refuses to fuse them and both mono halves are written instead.
### The AIFF twins

The thirteen ISO 9660 discs ship each sound twice, once as AIFF and once as WAV. 6 033 of the 7 498 AIFF hold audio that is already coming out as a WAV, and those are written once; the other 1 779 are converted, along with 314 whose AIFF carries a root key or a loop that its WAV twin does not. `vol.43` is the reason the check is on the audio and not the filename: its 1 386 AIFF all share a name with a WAV and **none of them share its audio**, being mastered a few frames longer ([ADR-0024](docs/adr/0024-the-aiff-twin-is-converted-and-deduplicated.md)).
The thirteen ISO 9660 ProSamples discs ship each sound twice, once as AIFF and once as WAV. 6 033 of the 7 498 AIFF hold audio that is already coming out as a WAV, and those are written once; the other 1 465 are converted, along with 314 whose AIFF carries a root key or a loop that its WAV twin does not. `vol.43` is the reason the check is on the audio and not the filename: its 1 386 AIFF all share a name with a WAV and **none of them share its audio**, being mastered a few frames longer ([ADR-0024](docs/adr/0024-the-aiff-twin-is-converted-and-deduplicated.md)).

These discs are also the only place in this project where the correct output is known independently. Where a sound exists as both, the publisher's own WAV says what the AIFF conversion should produce — including the loop convention the AIFF spec leaves open, settled on 195 pairs out of 195.

Expand Down
2 changes: 1 addition & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ All of that vanishes into a working parser. Six months on, the code says *what*
| D15 | Every partition of an AKAI disc, from the table it declares ([ADR-0023](adr/0023-partitions-come-from-the-table-the-disc-declares.md)) | done |
| D16 | AIFF payloads converted, deduplicated against their WAV twin, and EXS24/HALion instruments kept ([ADR-0024](adr/0024-the-aiff-twin-is-converted-and-deduplicated.md)) | done |

Across the local collection, by listing: 71 of 79 images claimed, 2 578 volumes, 110 989 files, 77 620 of them samples. The AKAI discs are 44 of those images and 68 997 of those files, read across 276 partitions — before D15 they were 14 670 files, because only the partition at the origin was read.
Across the local collection, by listing: 71 of 79 images claimed, 2 578 volumes, 110 989 files, 77 620 of them samples. The AKAI discs are 44 of those images and 68 997 of those files, read across 275 partitions — before D15 they were 14 670 files, because only the partition at the origin was read.

## What is not done

Expand Down
6 changes: 3 additions & 3 deletions docs/formats/akai-fs.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Getting index 10 wrong is the classic failure and it is not obvious: `KICKIN B0-
| `0x70A` | **block allocation map** — one u16 per block, as many as `0x00` declares |
| `0x4500` | **partition table** — the disk's own list of its partitions, in the first partition only |

`0x02`, `0xC6` and `0xC8` all hold on all **276 partitions** of the 44 AKAI discs, and together they are what identifies a partition header. The two size fields matter most: a block count with no echo behind it is not a partition's, which is a firmer test than anything about the image's length.
`0x02`, `0xC6` and `0xC8` all hold on all **275 partitions** of the 44 AKAI discs, and together they are what identifies a partition header. The two size fields matter most: a block count with no echo behind it is not a partition's, which is a firmer test than anything about the image's length.

**The pattern at `0x02` is a rising sawtooth, and sample data reproduces it.** As 16-bit PCM, `3333 × i` is a saw wave, so audio does match it — 374 blocks of one disc's free space carry a complete header prefix, every one in a block the allocation map calls free. That is why a partition header is *confirmed* where the table says one is, and never scanned for ([ADR-0023](../adr/0023-partitions-come-from-the-table-the-disc-declares.md)).

Expand Down Expand Up @@ -76,7 +76,7 @@ At `0x70A`, immediately after the volume directory's hundred slots: **one u16 pe

A file's extent is the chain from its start block to `0xC000`. **That is what verifies the map rather than merely making it plausible**: the chain length and the file size are stated by two different structures, and across all 44 AKAI discs they agree for **14 607 of 14 607 files** in the first partitions — exactly, with no disc disagreeing anywhere.

Read across all 276 partitions the figure is **68 267 of 68 284**, and the seventeen exceptions are one thing rather than a scatter: every one is a `MULTI FILE` — type `m`, all on `AKAI.S3000.Sound.Library.1` — whose chain runs exactly one block past what its size needs. A multi appears to be allocated a spare block. It is the only kind that disagrees anywhere.
Read across all 275 partitions the figure is **68 267 of 68 284**, and the seventeen exceptions are one thing rather than a scatter: every one is a `MULTI FILE` — type `m`, all on `AKAI.S3000.Sound.Library.1` — whose chain runs exactly one block past what its size needs. A multi appears to be allocated a spare block. It is the only kind that disagrees anywhere.

The exclusion behind that figure is itself the finding. Five volumes sit on blocks the map calls free, and their files' chains are gone with them; four of those five hold 63 files that read perfectly. These are **deleted volumes**: the blocks went back to the free list, and because the medium is a mastered CD-ROM nothing ever reused them, so the directory and the audio are still there to be read. They are listed with their files like any other volume.

Expand Down Expand Up @@ -119,7 +119,7 @@ A partition's own blocks are numbered from **its** start, so the same block numb

Where the image holds no header at a position the table declares, it is **skipped and not searched for**. On the discs where that happens the header turns up displaced by a whole number of the container's 32 KB blocks — the image is short of the disc it was made from, which is the fault below and not a filesystem to go hunting through.

Across the 44 discs the table places 276 partitions holding **2 154 volumes and 68 997 files**, against the 448 volumes and 14 670 files of the first partitions alone.
Across the 44 discs the table declares 384 partitions and 275 are present in the images, holding **2 154 volumes and 68 997 files**, against the 448 volumes and 14 670 files of the first partitions alone.

## An image can be short

Expand Down
6 changes: 3 additions & 3 deletions src/samplerdisc/fs/akai.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@
#: See docs/formats/akai-fs.md and ADR-0023.
PARTITION_TABLE_OFFSET = 0x4500

#: 196 bytes of constant at 0x02, byte-identical across all 276 partitions of
#: 196 bytes of constant at 0x02, byte-identical across all 275 partitions of
#: the 44 discs: 3333 x i as u16 LE, i = 0..97, wrapping. Nothing is known to
#: read it and nothing on any disc varies with it, but a partition header
#: carries it, which is what confirms a partition is where the table says.
Expand All @@ -130,7 +130,7 @@
HEADER_PATTERN = struct.pack("<98H", *(3333 * step & 0xFFFF for step in range(98)))

#: The header restates its own size. The u16 at 0xC6 is the block count at 0x00
#: plus this bias, and the u16 at 0xC8 is 47; both hold on all 276 partitions
#: plus this bias, and the u16 at 0xC8 is 47; both hold on all 275 partitions
#: measured. Two fields of the header agreeing is what lets a candidate be
#: confirmed by the disc rather than by the arithmetic that placed it.
SIZE_ECHO_OFFSET = 0xC6
Expand Down Expand Up @@ -269,7 +269,7 @@ def allocation_map(image: SectorImage, offset: int) -> list[int]:
from one, instead of a map invented out of whatever follows the header.

What makes a count sane is the header restating it: the u16 at 0xC6 is the
block count plus a fixed bias on all 276 partitions measured, so a count
block count plus a fixed bias on all 275 partitions measured, so a count
with no echo behind it is not a partition's and gets no map. That is a
firmer test than the image's length, which was the previous one and refused
the map to a partition the image merely *ends inside* -- leaving two volumes
Expand Down
Loading