Skip to content

Reduce payload decompression memory and abort on unpack failures - #7

Closed
Neofly1992 wants to merge 1 commit into
KindleModding:mainfrom
Neofly1992:codex/lower-payload-memory
Closed

Neofly1992 wants to merge 1 commit into
KindleModding:mainfrom
Neofly1992:codex/lower-payload-memory

Conversation

@Neofly1992

Copy link
Copy Markdown

Problem

On low-memory Kindles, the embedded payload's 64 MiB XZ dictionary can cause BusyBox to report xz: corrupted data. The generated script continues after extraction fails, potentially removing an existing KMC installation and eventually reporting success.

Addresses #5 and relates to KindleModding/Winterbreak2#8. Credit to @AbiLab-io for the original diagnosis and workaround, and to the users who confirmed it.

Changes

  • Compress the shared payload with --lzma2=preset=6,dict=12MiB, retaining one embedded payload for both platforms.
  • Check compression/base64 generation failures during the build.
  • Check every extraction pipeline stage, including XZ integrity errors, using pipefail confined to a subshell. Subsequent installation jobs retain their shell options.
  • Stop before installation if extraction fails or either platform's fbink / the system patch script is missing or empty. Clean up partial staging on failure.
  • Add eight regression tests for the generated prelude: valid extraction, shell-option isolation, damaged XZ footer, truncation, empty/missing-file archives, and missing build input.

Why 12 MiB?

Using the identical 24,330,240-byte tar from published jb.sh v2.2.1:

Configuration XZ size Decoder memory reported by xz
Official / 64 MiB dictionary 3,951,596 bytes 65 MiB
8 MiB / -6 (comparison run) 6,578,848 bytes 9 MiB
12 MiB (this packer) 3,951,448 bytes 13 MiB
16 MiB (comparison run) 3,949,852 bytes 17 MiB

Comparison streams and the final packer differ by a few bytes of XZ container metadata. The 8 MiB option adds about 3.50 MB of embedded base64; 12 MiB retains essentially the original size. This tradeoff is payload-dependent and should be rechecked as it changes. Decoder allocation estimates are not whole-process/device RAM measurements.

Validation and limitations

No physical Kindle testing has been performed, including on PW2. This is not a claim of a confirmed on-device fix. PW2 verification before merging would be welcome.

  • All eight tests pass with native ARM64 BusyBox 1.28.3 in Docker. On Linux with Python 3, xz, GNU base64 and BusyBox: python3 tests/test_payload_unpack.py -v.
  • The modified packer was run against the real published payload. Its generated unpack prelude succeeds with BusyBox 1.28.3 under a 64 MiB container limit, no swap, a 32 MiB /tmp tmpfs and ulimit -v 24000. All 38 extracted regular files match their original SHA256 hashes.
  • Earlier native tests reproduced the original payload's failure at ulimit -v 60000. Virtual-address limits do not represent available device RAM.
  • Repeated native container comparisons found essentially unchanged extraction time for 12 MiB versus the original. The complete pipeline uses more memory than the decoder alone.
  • ARMv7 emulated decoding was checked during investigation, but its memory limits were not representative and are not used as low-memory evidence.
  • The full cross-compilation/firmware build and installation/reboot stages were not run locally; only the packer and generated unpack prelude were executed.
  • The community workaround uses an external 8 MiB-dictionary payload, not this embedded 12 MiB proposal.

Prepared with Codex assistance; the validation above describes tests actually run.

@ThatHackerDudeFromCyberspace ThatHackerDudeFromCyberspace added the slop AI Generated Code That Doesn't Work Well label Oct 25, 2026
@ThatHackerDudeFromCyberspace

Copy link
Copy Markdown
Member

Hello, we understand that there is a decompression issue on certain Kindles and we are working to fix it on the affected models, unfortunately, this particular patch is illegible, clearly AI generated, and does not fully solve the problem.

As such this PR will be closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

slop AI Generated Code That Doesn't Work Well

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants