From ee5ead4387d8522cea2eb8cc41f305f50f1ab2c8 Mon Sep 17 00:00:00 2001 From: Geolm Date: Sun, 8 Nov 2026 17:41:17 -0500 Subject: [PATCH 1/2] flat index mode for blocks with identical endpoints (equal endpoints): a block whose two endpoint colors match carries a uniform index word (all 16 indices equal). a per-stream (BC1) / per-channel (BC4/BC5) flat index mode, decided once in the first pass, collapses it: mode 1 is selected when at least one block has identical endpoints and every such block has uniform indices, else mode 0. in mode 1 a flat block stores a single index value (2 bits for BC1, 3 bits per channel for BC4/BC5), expanded to all 16 indices on decode, instead of the top-table reference + mask + differences (BC1) or the 15 contextual symbols (BC4/BC5). the flag byte is written in the stream header (right after the top-table size for BC1, after the tile seeds for BC4/BC5) and is never needed for a non-flat block, so encoder and decoder always pick the same path; an encoder that leaves a non-uniform word on a flat block forces mode 0 and round-trips unchanged. the gpu decompressor mirrors the cpu; packed_*_maxsize grow by the alignment quantum to cover the header byte. new synthetic tests bc45_flat_uniform and bc3_flat_uniform exercise mode 1 (the existing flat tests keep mode 0). stream format is extended, hence the 1.0.0 version bump --- README.md | 20 +++- src/bc_decompress.metal | 78 +++++++++++---- src/bc_packed.c | 207 +++++++++++++++++++++++++++++++--------- src/bc_packed.h | 4 +- tests/synthetic_tests.c | 56 +++++++++++ 5 files changed, 294 insertions(+), 71 deletions(-) diff --git a/README.md b/README.md index 592fb99..e676e9d 100644 --- a/README.md +++ b/README.md @@ -198,6 +198,10 @@ For each set bit in the mask, the corresponding difference byte is encoded using This avoids storing zero bytes in the residual and makes the representation particularly compact when the block is close to one of the top-table patterns. +#### Flat blocks + +A block whose two endpoint colors are identical carries a uniform index word: all 16 indices hold the same 2-bit value. A global flat index mode, decided once in the first pass, collapses this: mode 1 is selected when at least one block has identical endpoints and every such block has a uniform index word, else mode 0. In mode 1 a flat block stores a single 2-bit value (expanded to all 16 indices on decode) instead of the top-table reference + mask + differences; in mode 0, or for a non-flat block, the indices are stored as usual. One byte of the mode is written in the stream header right after the top-table size. A decoder never needs the mode for a non-flat block, so the encoder and decoder always pick the same path. + ## Compressed stream Pseudo-description of the stream. @@ -211,7 +215,8 @@ The output buffer must be at least `packed_bc1_maxsize(width, height)` bytes. Th ```text * 19 static models, in order: for each endpoint (0, 1): red (3 context buckets), green (3 context buckets), blue (3 context buckets) — 16 color models — then top-table reference, difference mask, table difference. Each is 3 + num_symbols bytes, no padding (max 259 bytes). -* Top table size: one uint8_t (number of entries − 1), padded to a 4-byte boundary. +* Top table size: one uint8_t (number of entries − 1). +* Flat index mode: one uint8_t (0 or 1), then padded to a 4-byte boundary. See Block indices → Flat blocks. * Tile seed: one uint32_t (little-endian) — the two endpoint colors of the "previous block" used to predict the first block of each tile, averaged over the 64 tiles at compression time. Low 16 bits = color[0], high 16 bits = color[1]. * Tile predictor flags: 8 bytes (little-endian uint64_t) — one bit per tile: 1 → endpoint colors of that tile are predicted from the row above (same zigzag position) after the first scanline, 0 → zigzag-previous block. See Per-tile predictor. * Tile dictionary enable flags: 8 bytes (little-endian uint64_t) — one bit per tile: 1 → the tile's blocks write the color-pair dictionary flag + reference per block, 0 → the six color deltas only (byte-identical to the dictionary-less format). See Per-tile enable flag. @@ -233,9 +238,9 @@ Within a tile, the blocks are traversed in the zigzag row order. For each block: * If 0: the six color deltas, as below. * For a tile with the dictionary disabled: no flag. * Two endpoints, each: green, red, blue deltas (7-bit values offset by 64). -* Top-table reference (8-bit index). -* Difference mask (4 bits). -* One difference byte per set mask bit. +* If the block is flat (its two endpoint colors are identical) and the flat index mode is 1: + one 2-bit index value, expanded to all 16 indices on decode. +* Otherwise: top-table reference (8-bit index), difference mask (4 bits), and one difference byte per set mask bit. ``` See Color-pair dictionary. The six color deltas, the top-table reference, the mask, and the difference bytes are all encoded with Rice-Golomb using one of the 19 static models; each red/blue uses the model of its (channel, endpoint, context) — the three-bucket green delta described in Endpoints — and each green uses the model of its (endpoint, context), the first endpoint uncontexted and the second contextual on the sign of the first endpoint's green delta. The dictionary flag and reference are raw bits, not Rice-Golomb symbols. @@ -291,6 +296,10 @@ That is an 8 x 8 matrix = 64 index models per channel. The context is derived fr The 8-symbol models use a compact rank table (8 entries instead of 256): `3 + num_symbols ≤ 11` bytes each in the stream. +#### Flat blocks + +A block whose two endpoints are identical carries a uniform index word: all 16 indices hold the same 3-bit value. A per-channel flat index mode, decided once in the first pass, collapses this: mode 1 is selected for a channel when at least one block of that channel has identical endpoints and every such block has uniform indices, else mode 0. In mode 1 a flat block stores a single 3-bit value (expanded to all 16 indices on decode) instead of the 15 contextual symbols; in mode 0, or for a non-flat block, the indices are stored as usual. One byte of the mode per channel is written in the stream header after the tile seeds. A decoder never needs the mode for a non-flat block, so the encoder and decoder always pick the same path; an encoder that leaves a non-uniform index word on a flat block forces mode 0 and round-trips unchanged. + ### Compressed stream As BC1, the stream does not include width, height or format: the BC4 / BC5 decompressors are separate entry points and know their own header. @@ -299,13 +308,14 @@ As BC1, the stream does not include width, height or format: the BC4 / BC5 decom * 2 endpoint models per channel — 256-symbol static models, 3 + num_symbols bytes each, no padding (max 259). * 64 index models per channel — 8-symbol static models, 3 + num_symbols bytes each, no padding (max 11). * Tile seed: 2 bytes per channel (endpoint 0, endpoint 1). +* Flat index mode: one uint8_t (0 or 1) per channel. See Indices → Flat blocks. * Padded to a 2-byte boundary. * 63 tile offsets (uint16_t each), relative deltas in dword units. Tile 0 always starts the tile data; its offset is zero and not stored. * Padded to a 4-byte boundary. * 64 tile bitstreams, each 4-byte aligned, each flushed independently. ``` -For BC5 the red channel's models and seed come first, then the green channel's, then the shared offsets and tiles. Within a tile the blocks follow the zigzag row order; per block the payload is the red channel complete, then the green one, and per channel: endpoint 0, endpoint 1, index 0 (3 raw bits), then indices 1..15. +For BC5 the red channel's models and seed come first, then the green channel's, then the shared offsets and tiles. Within a tile the blocks follow the zigzag row order; per block the payload is the red channel complete, then the green one, and per channel: endpoint 0, endpoint 1, then either one 3-bit index value (a flat block in mode 1, expanded to all 16 indices) or index 0 (3 raw bits) followed by indices 1..15 (each XORed with its intra-block prediction, as above). ### Status diff --git a/src/bc_decompress.metal b/src/bc_decompress.metal index 738dd98..5843d85 100644 --- a/src/bc_decompress.metal +++ b/src/bc_decompress.metal @@ -25,10 +25,17 @@ // buffer(2): constant decode_args_t // // The stream layout parsed here mirrors the CPU compressors / decompressors: -// BC1 [19 static models][top table size u8][4-align][strip seed u32][strip predictor u64][strip dict enable u64] -// [top table u32s][2-align][63 strip offsets u16, deltas in dword units][4-align][strip bitstreams] +// BC1 [19 static models][top table size u8][flat index mode u8][4-align][strip seed u32] +// [strip predictor u64][strip dict enable u64][top table u32s][2-align][63 strip offsets u16, +// deltas in dword units][4-align][strip bitstreams] // BC4 / BC5 [2 endpoint models per channel][64 index models per channel][strip seed, 2 bytes per channel] -// [2-align][63 strip offsets u16, deltas in dword units][4-align][strip bitstreams] +// [flat index mode, 1 byte per channel][2-align][63 strip offsets u16, deltas in dword units] +// [4-align][strip bitstreams] +// +// The flat index mode (0 / 1) is decided once per stream (per channel for BC4 / BC5) in the first pass: +// 1 when at least one block has identical endpoints and every such block has uniform indices. A block +// with identical endpoints then stores a single index value (2 bits for BC1, 3 bits per channel for +// BC4 / BC5) expanded to all 16 indices, instead of the full index payload. // // On a corrupted stream the decode runs to completion and the output is garbage, mirroring the // CPU version which only reports the failure after the whole strip was decoded. A valid stream @@ -391,6 +398,9 @@ kernel void bcp_decompress_bc1(const device uint8_t* input [[buffer(0)]], return; uint top_table_size = input[pos++] + 1; + if (pos + 1 > length) + return; + const bool flat_index_mode = input[pos++] != 0; // flat-index mode, right after the top-table size pos = (pos + 3) & ~(uint64_t)3; // 4-byte align if (pos + 20 > length) @@ -541,16 +551,27 @@ kernel void bcp_decompress_bc1(const device uint8_t* input [[buffer(0)]], } } - // indices difference with top table - uint8_t reference = decode_symbol(&compressed_stream, &reference_model); - uint8_t mask = decode_symbol(&compressed_stream, &mask_model); + // a flat block (identical endpoints, reconstructed exactly like the encoder sees the source) in + // mode 1 stores one 2-bit value, expanded to all 16 indices; otherwise the full reference + + // mask + differences + uint32_t indices; + if ((current_color0 == current_color1) && flat_index_mode) + { + uint8_t value = bs_read_bits(&compressed_stream, 2); + indices = (uint32_t)value * 0x11111111u; + } + else + { + uint8_t reference = decode_symbol(&compressed_stream, &reference_model); + uint8_t mask = decode_symbol(&compressed_stream, &mask_model); - uint32_t difference = 0; - for (uint j = 0; j < 4; ++j) - if (mask & (1 << j)) - difference |= (uint32_t)decode_symbol(&compressed_stream, &difference_model) << (j * 8); + uint32_t difference = 0; + for (uint j = 0; j < 4; ++j) + if (mask & (1 << j)) + difference |= (uint32_t)decode_symbol(&compressed_stream, &difference_model) << (j * 8); - uint32_t indices = difference ^ top_table[reference]; + indices = difference ^ top_table[reference]; + } write_block(output, block_index, current_color0, current_color1, indices); @@ -640,6 +661,14 @@ void bc45_decode_strip(const device uint8_t* input, seed[c][e] = input[pos++]; } + // flat-index mode: one byte per channel, after the seeds, before the 2-align + if (pos + NumChannels > length) + return; + + bool flat_mode[NumChannels]; + for (uint c = 0; c < NumChannels; ++c) + flat_mode[c] = input[pos++] != 0; + //------------------------------------------------------------------------------------------------------------------------- // strip offsets (deltas in dword units), then the strip bitstream of this thread pos = (pos + 1) & ~(uint64_t)1; // 2-byte align @@ -721,16 +750,27 @@ void bc45_decode_strip(const device uint8_t* input, // index payload: the first index in raw 3 bits, then each index xored with its // intra-block prediction, contextual on the 8 x 8 [left][top] matrix - idx[0] = bs_read_bits(&compressed_stream, 3); - - for (uint i = 1; i < 16; ++i) + // a flat block (identical endpoints, reconstructed exactly like the encoder sees the + // source) in mode 1 stores one 3-bit value, expanded to all 16 indices + if ((e[0] == e[1]) && flat_mode[c]) { - // the missing neighbor of the pixel (first row, row start) defaults to 0 - uint8_t left = ((i & 3) == 0) ? 0 : idx[i - 1]; - uint8_t top = (i < 4) ? 0 : idx[i - 4]; - uint8_t p = ((i & 3) == 0) ? top : left; + uint8_t value = bs_read_bits(&compressed_stream, 3); + for (uint i = 0; i < 16; ++i) + idx[i] = value; + } + else + { + idx[0] = bs_read_bits(&compressed_stream, 3); + + for (uint i = 1; i < 16; ++i) + { + // the missing neighbor of the pixel (first row, row start) defaults to 0 + uint8_t left = ((i & 3) == 0) ? 0 : idx[i - 1]; + uint8_t top = (i < 4) ? 0 : idx[i - 4]; + uint8_t p = ((i & 3) == 0) ? top : left; - idx[i] = (uint8_t)(p ^ decode_symbol8(&compressed_stream, &index[c][bc4_index_context(left, top)])); + idx[i] = (uint8_t)(p ^ decode_symbol8(&compressed_stream, &index[c][bc4_index_context(left, top)])); + } } // the standard block: two endpoints + the 48-bit little-endian index field, as two aligned u32 words diff --git a/src/bc_packed.c b/src/bc_packed.c index 69fd154..800fcab 100644 --- a/src/bc_packed.c +++ b/src/bc_packed.c @@ -540,12 +540,28 @@ struct bcp_context // matching blocks) pay no flag overhead. see the per-strip dictionary enable in init_static_models uint64_t strip_dict_enable; + // flat-index mode of the whole stream: 1 → a block whose two endpoints are identical stores a single + // 2-bit index value, expanded to all 16 indices, instead of the reference + mask + differences; + // 0 → every block stores its indices as usual. decided once in the first pass: 1 only if at least + // one block has identical endpoints and every such block has a uniform index word (v * 0x11111111), + // else 0 (an encoder that leaves garbage indices on a flat block forces mode 0). the flag is written + // in the stream header's alignment padding, absorbed except in the worst parity case (a few bytes, + // covered by packed_bc1_maxsize) + uint8_t bc1_index_mode; + // bc4/bc5: per-channel static models — the 2 endpoint models (256 symbols) and the 64 // index models (8 symbols each, 8 x 8 [left][top] matrix); each retains the full 516-byte // le_model in memory, the stream only stores num_symbols entries le_model bc4_endpoint_model[BCP_MAX_CHANNELS][2]; // [channel][endpoint] le_model bc4_index_model[BCP_MAX_CHANNELS][BC4_INDEX_CONTEXTS]; uint8_t bc4_strip_seed[BCP_MAX_CHANNELS][2]; // [channel][endpoint] + + // per-channel flat-index mode: 1 → a block whose two endpoints are identical stores a single 3-bit + // index value, expanded to all 16 indices, instead of the 15 contextual symbols; 0 → the indices + // are stored as usual. decided per channel in the first pass, same rule as the BC1 mode. one byte + // per channel in the stream header after the seeds, in the 2-byte alignment padding (worst case a + // couple bytes, covered by packed_bc4_maxsize / packed_bc5_maxsize) + uint8_t bc4_index_mode[BCP_MAX_CHANNELS]; // [channel] }; typedef struct bc1_block @@ -1252,6 +1268,11 @@ void init_static_models(bcp_context* ctx, const void* input, uint32_t width_bloc // symbols (the dictionary does not touch the indices). the color models fit on this pass price // the matched blocks in the per-strip dictionary decision below, the final color models are // refit on the deltas that are actually emitted after the decision + // the flat-index mode is decided here too: mode 1 (a block with identical endpoints stores one + // 2-bit index) only if at least one block has identical endpoints and every such block has a + // uniform index word, else mode 0 (garbage indices on a flat block force the full encoding) + bool flat_block = false; + bool non_uniform_flat = false; for(uint32_t strip_index=0; strip_indexcolor[0] == current->color[1]) + { + flat_block = true; + const uint8_t value = (uint8_t)(current->indices & 3); + if (current->indices != (uint32_t)value * 0x11111111u) + non_uniform_flat = true; + } + uint8_t reference = nearest32(ctx->top_table, ctx->top_table_size, current->indices) & 0xff; hist_reference[reference]++; @@ -1298,6 +1327,8 @@ void init_static_models(bcp_context* ctx, const void* input, uint32_t width_bloc } } + ctx->bc1_index_mode = (flat_block && !non_uniform_flat) ? 1 : 0; + init_color_models(ctx, &hist); // per-strip color-pair dictionary enable: an enabled strip pays one flag bit per block, so it @@ -1507,12 +1538,18 @@ typedef struct bc4_hists //----------------------------------------------------------------------------------------------------------------------------- // Fits the bc4/bc5 static models from [input]: the per-channel tile seed is the average of the // first block of each tile in chain order (as BC1), then one histogram pass with the exact -// tile predictors of the encoder. there is no per-tile mode decision, so a single fit is final +// tile predictors of the encoder. there is no per-tile mode decision, so a single fit is final. +// the per-channel flat-index mode is also decided in the histogram pass: mode 1 (a block whose +// two endpoints are identical stores one 3-bit index) only if at least one such block exists and +// every such block has uniform indices, else mode 0 (garbage indices on a flat block force the +// full 15-symbol encoding) static void bc4_init_models(bcp_context* ctx, const void* image, uint32_t width_blocks, uint32_t height_blocks, uint32_t num_strips, uint32_t num_channels, uint32_t block_stride) { bc4_hists hist[BCP_MAX_CHANNELS] = { 0 }; uint32_t sums[BCP_MAX_CHANNELS][2] = { 0 }; uint8_t prev_e[BCP_MAX_CHANNELS][2], top_e[BCP_MAX_CHANNELS][2], top_prev_e[BCP_MAX_CHANNELS][2]; + bool flat_block[BCP_MAX_CHANNELS] = { false, false }; + bool non_uniform_flat[BCP_MAX_CHANNELS] = { false, false }; // pass 1: tile seeds, the average of the first block of each tile in chain order for(uint32_t strip_index=0; strip_indexbc4_strip_seed[c][e] = (uint8_t)(sums[c][e] / num_strips); } - // pass 2: histograms with the exact tile predictors of the encoder + // pass 2: histograms with the exact tile predictors of the encoder, plus the per-channel + // flat-index mode decision (see the function header) for(uint32_t strip_index=0; strip_indexendpoint[0] == b->endpoint[1]) + { + flat_block[c] = true; + const uint8_t first = idx[0]; + for(uint32_t i=0; i<16; ++i) + { + if (idx[i] != first) + { + non_uniform_flat[c] = true; + break; + } + } + } + for(uint32_t i=1; i<16; ++i) { // the missing neighbor of the pixel (first row, row start) defaults to 0 @@ -1604,6 +1656,9 @@ static void bc4_init_models(bcp_context* ctx, const void* image, uint32_t width_ } } + for(uint32_t c=0; cbc4_index_mode[c] = (flat_block[c] && !non_uniform_flat[c]) ? 1 : 0; + for(uint32_t c=0; cbc4_strip_seed[c][e]; } + // flat-index mode, one byte per channel, written right after the seeds. the model region is always + // even-sized, so this byte sits on an odd stream position and the 2-align below can never land on + // it and zero it, keeping the encoder and the decoder in agreement + if (stream.pos + num_channels > stream.length) + return 0; + for(uint32_t c=0; cbc4_index_mode[c]; + stream_align(&stream, sizeof(uint16_t)); size_t strips_offset_array_size = sizeof(uint16_t) * (*num_strips - 1); @@ -1725,16 +1788,24 @@ static size_t bcp_compress_bc45(bcp_context* ctx, const void* image, uint32_t wi // intra-block prediction, contextual on the 8 x 8 [left][top] matrix bc4_unpack_indices(b, idx); - le_write_bits(&compressed_stream, idx[0], 3); - - for(uint32_t i=1; i<16; ++i) + // a flat block (identical endpoints) in mode 1 stores one 3-bit value, expanded to + // all 16 indices on decode; otherwise the full 15 contextual symbols + const bool flat = (b->endpoint[0] == b->endpoint[1]); + if (flat && ctx->bc4_index_mode[c]) + le_write_bits(&compressed_stream, idx[0], 3); + else { - // the missing neighbor of the pixel (first row, row start) defaults to 0 - uint8_t left = ((i & 3) == 0) ? 0 : idx[i-1]; - uint8_t top = (i < 4) ? 0 : idx[i-4]; - uint8_t p = ((i & 3) == 0) ? top : left; + le_write_bits(&compressed_stream, idx[0], 3); + + for(uint32_t i=1; i<16; ++i) + { + // the missing neighbor of the pixel (first row, row start) defaults to 0 + uint8_t left = ((i & 3) == 0) ? 0 : idx[i-1]; + uint8_t top = (i < 4) ? 0 : idx[i-4]; + uint8_t p = ((i & 3) == 0) ? top : left; - le_encode_symbol(&compressed_stream, &ctx->bc4_index_model[c][bc4_index_context(left, top)], p ^ idx[i]); + le_encode_symbol(&compressed_stream, &ctx->bc4_index_model[c][bc4_index_context(left, top)], p ^ idx[i]); + } } if (!first_row) @@ -1795,6 +1866,11 @@ static bool bcp_decompress_bc45(bcp_context* ctx, const void* input, size_t inpu ctx->bc4_strip_seed[c][e] = stream.buffer[stream.pos++]; } + if (stream.pos + num_channels > stream.length) + return false; + for(uint32_t c=0; cbc4_index_mode[c] = stream.buffer[stream.pos++]; // flat-index mode, after the seeds (see bc4_index_mode) + read_stream_align(&stream, sizeof(uint16_t)); size_t strips_offset_array_size = sizeof(uint16_t) * (num_strips - 1); @@ -1862,15 +1938,27 @@ static bool bcp_decompress_bc45(bcp_context* ctx, const void* input, size_t inpu // index payload: the first index in raw 3 bits, then each index xored with its // intra-block prediction, contextual on the 8 x 8 [left][top] matrix - idx[0] = le_read_bits(&compressed_stream, 3); - - for(uint32_t i=1; i<16; ++i) + // a flat block (identical endpoints, reconstructed exactly like the encoder sees the + // source) in mode 1 stores one 3-bit value, expanded to all 16 indices + const bool flat = (b->endpoint[0] == b->endpoint[1]); + if (flat && ctx->bc4_index_mode[c]) + { + const uint8_t value = le_read_bits(&compressed_stream, 3); + for(uint32_t i=0; i<16; ++i) + idx[i] = value; + } + else { - uint8_t left = ((i & 3) == 0) ? 0 : idx[i-1]; - uint8_t top = (i < 4) ? 0 : idx[i-4]; - uint8_t p = ((i & 3) == 0) ? top : left; + idx[0] = le_read_bits(&compressed_stream, 3); - idx[i] = (uint8_t)(p ^ le_decode_symbol(&compressed_stream, &ctx->bc4_index_model[c][bc4_index_context(left, top)])); + for(uint32_t i=1; i<16; ++i) + { + uint8_t left = ((i & 3) == 0) ? 0 : idx[i-1]; + uint8_t top = (i < 4) ? 0 : idx[i-4]; + uint8_t p = ((i & 3) == 0) ? top : left; + + idx[i] = (uint8_t)(p ^ le_decode_symbol(&compressed_stream, &ctx->bc4_index_model[c][bc4_index_context(left, top)])); + } } bc4_pack_indices(idx, b); @@ -1912,8 +2000,10 @@ bcp_context* bcp_init(bcp_mem_interface* user_mem) //----------------------------------------------------------------------------------------------------------------------------- size_t packed_bc1_maxsize(uint32_t width, uint32_t height) { - // very rough - return (width/4) * (height/4) * 2 * sizeof(bc1_block); + // very rough. the +4 covers the worst-case header growth of the flat-index mode byte: it is written + // right after the top-table size, before the 4-byte align, so the header can grow by up to the + // 4-byte alignment quantum + return (width/4) * (height/4) * 2 * sizeof(bc1_block) + 4; } //----------------------------------------------------------------------------------------------------------------------------- @@ -1953,11 +2043,12 @@ static size_t bcp_compress_bc1_impl(bcp_context* ctx, const void* bc1_image, uin if (!save_static_model(&ctx->mask_model, &stream)) return 0; if (!save_static_model(&ctx->table_difference_model, &stream)) return 0; - // top-table + strip seed + predictor flags + dictionary enable flags - if (stream.pos + 1 + 3 + sizeof(uint32_t) + 8 + 8 + ctx->top_table_size * sizeof(uint32_t) > stream.length) + // top-table + flat-index mode + strip seed + predictor flags + dictionary enable flags + if (stream.pos + 1 + 1 + 3 + sizeof(uint32_t) + 8 + 8 + ctx->top_table_size * sizeof(uint32_t) > stream.length) return 0; stream.buffer[stream.pos++] = (uint8_t) (ctx->top_table_size - 1); // there is no zero toptable, so minus 1 to fit in a uint8_t + stream.buffer[stream.pos++] = ctx->bc1_index_mode; // flat-index mode, written in the alignment padding (see bc1_index_mode) stream_align(&stream, sizeof(uint32_t)); for(uint32_t j=0; j<4; ++j) @@ -2069,22 +2160,31 @@ static size_t bcp_compress_bc1_impl(bcp_context* ctx, const void* bc1_image, uin le_encode_symbol(&compressed_stream, &ctx->blue_model[j][chroma_context], dblue + COLOR_DELTA_OFFSET); } } - uint8_t reference = nearest32(ctx->top_table, ctx->top_table_size, current->indices) & 0xff; - le_encode_symbol(&compressed_stream, &ctx->table_reference_model, reference); + // a flat block (identical endpoints) in mode 1 stores one 2-bit value, expanded to all + // 16 indices on decode; otherwise the full reference + mask + differences + const bool flat = (current->color[0] == current->color[1]); + if (flat && ctx->bc1_index_mode) + le_write_bits(&compressed_stream, current->indices & 3, 2); + else + { + uint8_t reference = nearest32(ctx->top_table, ctx->top_table_size, current->indices) & 0xff; - uint32_t difference = current->indices ^ ctx->top_table[reference]; - uint32_t mask = 0; - if ((difference & 0x000000FF) != 0) mask |= 1; - if ((difference & 0x0000FF00) != 0) mask |= 2; - if ((difference & 0x00FF0000) != 0) mask |= 4; - if ((difference & 0xFF000000) != 0) mask |= 8; + le_encode_symbol(&compressed_stream, &ctx->table_reference_model, reference); - le_encode_symbol(&compressed_stream, &ctx->mask_model, mask); + uint32_t difference = current->indices ^ ctx->top_table[reference]; + uint32_t mask = 0; + if ((difference & 0x000000FF) != 0) mask |= 1; + if ((difference & 0x0000FF00) != 0) mask |= 2; + if ((difference & 0x00FF0000) != 0) mask |= 4; + if ((difference & 0xFF000000) != 0) mask |= 8; - for(uint32_t j=0; j<4; ++j) - if (mask & (1u << j)) - le_encode_symbol(&compressed_stream, &ctx->table_difference_model, (difference >> (j*8)) & 0xff); + le_encode_symbol(&compressed_stream, &ctx->mask_model, mask); + + for(uint32_t j=0; j<4; ++j) + if (mask & (1u << j)) + le_encode_symbol(&compressed_stream, &ctx->table_difference_model, (difference >> (j*8)) & 0xff); + } dict.entry[position & COLOR_DICT_MASK] = pair; previous = *current; @@ -2144,6 +2244,9 @@ static bool bcp_decompress_bc1_impl(bcp_context* ctx, const void* input, size_t return false; ctx->top_table_size = (uint32_t)stream.buffer[stream.pos++] + 1; + if (stream.pos + 1 > stream.length) + return false; + ctx->bc1_index_mode = stream.buffer[stream.pos++]; // flat-index mode, written right after the top-table size (see bc1_index_mode) read_stream_align(&stream, sizeof(uint32_t)); if (stream.pos + sizeof(uint32_t) + 8 + 8 > stream.length) @@ -2261,16 +2364,27 @@ static bool bcp_decompress_bc1_impl(bcp_context* ctx, const void* input, size_t } } - // indices difference with top table - uint32_t reference = le_decode_symbol(&compressed_stream, &ctx->table_reference_model); - uint32_t mask = le_decode_symbol(&compressed_stream, &ctx->mask_model); + // a flat block (identical endpoints, reconstructed exactly like the encoder sees the source) in + // mode 1 stores one 2-bit value, expanded to all 16 indices; otherwise the full reference + + // mask + differences + const bool flat = (current->color[0] == current->color[1]); + if (flat && ctx->bc1_index_mode) + { + const uint8_t value = le_read_bits(&compressed_stream, 2); + current->indices = (uint32_t)value * 0x11111111u; + } + else + { + uint32_t reference = le_decode_symbol(&compressed_stream, &ctx->table_reference_model); + uint32_t mask = le_decode_symbol(&compressed_stream, &ctx->mask_model); - uint32_t difference=0; - for(uint32_t j=0; j<4; ++j) - if (mask & (1 << j)) - difference = difference | ((uint32_t)le_decode_symbol(&compressed_stream, &ctx->table_difference_model) << (j*8)); + uint32_t difference=0; + for(uint32_t j=0; j<4; ++j) + if (mask & (1 << j)) + difference = difference | ((uint32_t)le_decode_symbol(&compressed_stream, &ctx->table_difference_model) << (j*8)); - current->indices = difference ^ ctx->top_table[reference]; + current->indices = difference ^ ctx->top_table[reference]; + } dict.entry[position & COLOR_DICT_MASK] = color_pair(current); previous = *current; @@ -2289,15 +2403,18 @@ bool bcp_decompress_bc1(bcp_context* ctx, const void* input, size_t input_length //----------------------------------------------------------------------------------------------------------------------------- size_t packed_bc4_maxsize(uint32_t width, uint32_t height) { - // very rough: 2x the standard BC4 block size, the BC1 convention - return (width/4) * (height/4) * 2 * sizeof(bc4_block); + // very rough: 2x the standard BC4 block size, the BC1 convention. the +2 covers the worst-case + // header growth of the flat-index mode byte (one byte, before the 2-byte align = the align quantum) + return (width/4) * (height/4) * 2 * sizeof(bc4_block) + 2; } //----------------------------------------------------------------------------------------------------------------------------- size_t packed_bc5_maxsize(uint32_t width, uint32_t height) { - // very rough: 2x the standard BC5 block size (a red + a green bc4_block) - return (width/4) * (height/4) * 2 * (2 * sizeof(bc4_block)); + // very rough: 2x the standard BC5 block size (a red + a green bc4_block). the +3 covers the + // worst-case header growth of the flat-index mode bytes (2 bytes, before the 2-byte align, plus + // the alignment itself in the worst parity case) + return (width/4) * (height/4) * 2 * (2 * sizeof(bc4_block)) + 3; } //----------------------------------------------------------------------------------------------------------------------------- diff --git a/src/bc_packed.h b/src/bc_packed.h index c68840a..8792e55 100644 --- a/src/bc_packed.h +++ b/src/bc_packed.h @@ -12,9 +12,9 @@ extern "C" { //---------------------------------------------------------------------------------------------------------------------------- // version -#define BC_PACKED_MAJOR 0 +#define BC_PACKED_MAJOR 1 #define BC_PACKED_MINOR 0 -#define BC_PACKED_PATCH 1 +#define BC_PACKED_PATCH 0 //---------------------------------------------------------------------------------------------------------------------------- diff --git a/tests/synthetic_tests.c b/tests/synthetic_tests.c index d448da9..347e989 100644 --- a/tests/synthetic_tests.c +++ b/tests/synthetic_tests.c @@ -656,6 +656,34 @@ TEST bc45_flat(void) return bc45_both(image, image16, TEST_BC45_WIDTH, TEST_BC45_HEIGHT); } +//----------------------------------------------------------------------------------------------------------------------------- +// one solid block value per channel with UNIFORM indices: a flat block (identical endpoints) whose +// index word is constant selects the single-index flat-index mode (mode 1) for the channel, unlike +// bc45_flat whose non-uniform indices force mode 0. each channel uses a different value +TEST bc45_flat_uniform(void) +{ + bc4_block image[TEST_BC45_BLOCKS]; + bc4_block image16[TEST_BC45_BLOCKS * 2]; // 16-byte BC5 block: the red bc4_block followed by the green one + + for(uint32_t i=0; i Date: Sun, 8 Nov 2026 19:47:44 -0500 Subject: [PATCH 2/2] whole-image flat index value for blocks with identical endpoints: instead of storing one index value per flat block, the value now lives once in the stream header. the first pass requires every equal-endpoint block to hold the same uniform index value (2-bit for BC1, 3-bit per channel for BC4/BC5); a single shared value selects mode 1, in which a flat block stores no index payload at all and the decoder expands the header value to all 16 indices. an encoder that leaves a non-uniform or a differing word on a flat block forces mode 0, the classic per-block encoding, so the roundtrip stays byte-exact either way. the mode and the value share the existing header byte (mode | value << 1, right after the top-table size for BC1, one per channel after the tile seeds for BC4/BC5), so packed_*_maxsize are unchanged. the gpu decompressor mirrors the cpu. new synthetic tests: identical_colors_uniform_value / _divergent / _garbage / _mixed (BC1) and bc45_flat_uniform_word / _divergent (BC4+BC5); bc45_flat_uniform is re-documented, its 0x33 word is 6,3,1 repeating as 3-bit values (mode 0 all along), and bc45_flat_uniform_word with a truly uniform word is the real mode 1 coverage. 1.0.0 was never published, hence the 1.0.1 patch bump. release and asan builds: 43/43 tests, gpu bit-identical to cpu --- README.md | 12 +-- src/bc_decompress.metal | 40 ++++---- src/bc_packed.c | 138 +++++++++++++++++---------- src/bc_packed.h | 2 +- tests/synthetic_tests.c | 203 +++++++++++++++++++++++++++++++++++++++- 5 files changed, 319 insertions(+), 76 deletions(-) diff --git a/README.md b/README.md index e676e9d..a65b87f 100644 --- a/README.md +++ b/README.md @@ -200,7 +200,7 @@ This avoids storing zero bytes in the residual and makes the representation part #### Flat blocks -A block whose two endpoint colors are identical carries a uniform index word: all 16 indices hold the same 2-bit value. A global flat index mode, decided once in the first pass, collapses this: mode 1 is selected when at least one block has identical endpoints and every such block has a uniform index word, else mode 0. In mode 1 a flat block stores a single 2-bit value (expanded to all 16 indices on decode) instead of the top-table reference + mask + differences; in mode 0, or for a non-flat block, the indices are stored as usual. One byte of the mode is written in the stream header right after the top-table size. A decoder never needs the mode for a non-flat block, so the encoder and decoder always pick the same path. +A block whose two endpoint colors are identical carries a uniform index word: all 16 indices hold the same 2-bit value. A global flat index mode, decided once in the first pass, collapses this: mode 1 is selected when at least one block has identical endpoints and every such block holds the same 2-bit value. In mode 1 a flat block stores no index payload at all: the value, stored once for the whole image in the stream header, is expanded to all 16 indices on decode, instead of the top-table reference + mask + differences; in mode 0, or for a non-flat block, the indices are stored as usual. The mode and the value share one header byte (mode | value << 1) right after the top-table size. A decoder never needs the mode for a non-flat block, so the encoder and decoder always pick the same path; an encoder that leaves a non-uniform or a differing index word on a flat block forces mode 0 and round-trips byte-exact. ## Compressed stream @@ -216,7 +216,7 @@ The output buffer must be at least `packed_bc1_maxsize(width, height)` bytes. Th * 19 static models, in order: for each endpoint (0, 1): red (3 context buckets), green (3 context buckets), blue (3 context buckets) — 16 color models — then top-table reference, difference mask, table difference. Each is 3 + num_symbols bytes, no padding (max 259 bytes). * Top table size: one uint8_t (number of entries − 1). -* Flat index mode: one uint8_t (0 or 1), then padded to a 4-byte boundary. See Block indices → Flat blocks. +* Flat index mode + value: one uint8_t (mode | value << 1), then padded to a 4-byte boundary. See Block indices → Flat blocks. * Tile seed: one uint32_t (little-endian) — the two endpoint colors of the "previous block" used to predict the first block of each tile, averaged over the 64 tiles at compression time. Low 16 bits = color[0], high 16 bits = color[1]. * Tile predictor flags: 8 bytes (little-endian uint64_t) — one bit per tile: 1 → endpoint colors of that tile are predicted from the row above (same zigzag position) after the first scanline, 0 → zigzag-previous block. See Per-tile predictor. * Tile dictionary enable flags: 8 bytes (little-endian uint64_t) — one bit per tile: 1 → the tile's blocks write the color-pair dictionary flag + reference per block, 0 → the six color deltas only (byte-identical to the dictionary-less format). See Per-tile enable flag. @@ -239,7 +239,7 @@ Within a tile, the blocks are traversed in the zigzag row order. For each block: * For a tile with the dictionary disabled: no flag. * Two endpoints, each: green, red, blue deltas (7-bit values offset by 64). * If the block is flat (its two endpoint colors are identical) and the flat index mode is 1: - one 2-bit index value, expanded to all 16 indices on decode. + no index payload, the header value is expanded to all 16 indices on decode. * Otherwise: top-table reference (8-bit index), difference mask (4 bits), and one difference byte per set mask bit. ``` @@ -298,7 +298,7 @@ The 8-symbol models use a compact rank table (8 entries instead of 256): `3 + nu #### Flat blocks -A block whose two endpoints are identical carries a uniform index word: all 16 indices hold the same 3-bit value. A per-channel flat index mode, decided once in the first pass, collapses this: mode 1 is selected for a channel when at least one block of that channel has identical endpoints and every such block has uniform indices, else mode 0. In mode 1 a flat block stores a single 3-bit value (expanded to all 16 indices on decode) instead of the 15 contextual symbols; in mode 0, or for a non-flat block, the indices are stored as usual. One byte of the mode per channel is written in the stream header after the tile seeds. A decoder never needs the mode for a non-flat block, so the encoder and decoder always pick the same path; an encoder that leaves a non-uniform index word on a flat block forces mode 0 and round-trips unchanged. +A block whose two endpoints are identical carries a uniform index word: all 16 indices hold the same 3-bit value. A per-channel flat index mode, decided once in the first pass, collapses this: mode 1 is selected for a channel when at least one block of that channel has identical endpoints and every such block holds the same 3-bit value, else mode 0. In mode 1 a flat block stores no index payload at all: the value, stored once per channel for the whole image in the stream header, is expanded to all 16 indices on decode, instead of the 15 contextual symbols; in mode 0, or for a non-flat block, the indices are stored as usual. The mode and the value share one header byte per channel (mode | value << 1) after the tile seeds. A decoder never needs the mode for a non-flat block, so the encoder and decoder always pick the same path; an encoder that leaves a non-uniform or a differing index word on a flat block forces mode 0 and round-trips byte-exact. ### Compressed stream @@ -308,14 +308,14 @@ As BC1, the stream does not include width, height or format: the BC4 / BC5 decom * 2 endpoint models per channel — 256-symbol static models, 3 + num_symbols bytes each, no padding (max 259). * 64 index models per channel — 8-symbol static models, 3 + num_symbols bytes each, no padding (max 11). * Tile seed: 2 bytes per channel (endpoint 0, endpoint 1). -* Flat index mode: one uint8_t (0 or 1) per channel. See Indices → Flat blocks. +* Flat index mode + value: one uint8_t (mode | value << 1) per channel. See Indices → Flat blocks. * Padded to a 2-byte boundary. * 63 tile offsets (uint16_t each), relative deltas in dword units. Tile 0 always starts the tile data; its offset is zero and not stored. * Padded to a 4-byte boundary. * 64 tile bitstreams, each 4-byte aligned, each flushed independently. ``` -For BC5 the red channel's models and seed come first, then the green channel's, then the shared offsets and tiles. Within a tile the blocks follow the zigzag row order; per block the payload is the red channel complete, then the green one, and per channel: endpoint 0, endpoint 1, then either one 3-bit index value (a flat block in mode 1, expanded to all 16 indices) or index 0 (3 raw bits) followed by indices 1..15 (each XORed with its intra-block prediction, as above). +For BC5 the red channel's models and seed come first, then the green channel's, then the shared offsets and tiles. Within a tile the blocks follow the zigzag row order; per block the payload is the red channel complete, then the green one, and per channel: endpoint 0, endpoint 1, then either no index payload (a flat block in mode 1, the header value is expanded to all 16 indices on decode) or index 0 (3 raw bits) followed by indices 1..15 (each XORed with its intra-block prediction, as above). ### Status diff --git a/src/bc_decompress.metal b/src/bc_decompress.metal index 5843d85..999b834 100644 --- a/src/bc_decompress.metal +++ b/src/bc_decompress.metal @@ -25,17 +25,18 @@ // buffer(2): constant decode_args_t // // The stream layout parsed here mirrors the CPU compressors / decompressors: -// BC1 [19 static models][top table size u8][flat index mode u8][4-align][strip seed u32] +// BC1 [19 static models][top table size u8][flat index mode + value u8][4-align][strip seed u32] // [strip predictor u64][strip dict enable u64][top table u32s][2-align][63 strip offsets u16, // deltas in dword units][4-align][strip bitstreams] // BC4 / BC5 [2 endpoint models per channel][64 index models per channel][strip seed, 2 bytes per channel] -// [flat index mode, 1 byte per channel][2-align][63 strip offsets u16, deltas in dword units] +// [flat index mode + value, 1 byte per channel][2-align][63 strip offsets u16, deltas in dword units] // [4-align][strip bitstreams] // -// The flat index mode (0 / 1) is decided once per stream (per channel for BC4 / BC5) in the first pass: -// 1 when at least one block has identical endpoints and every such block has uniform indices. A block -// with identical endpoints then stores a single index value (2 bits for BC1, 3 bits per channel for -// BC4 / BC5) expanded to all 16 indices, instead of the full index payload. +// The flat index mode (bit 0 of the mode + value byte, the value in the upper bits) is decided once per +// stream (per channel for BC4 / BC5) in the first pass: 1 when at least one block has identical +// endpoints and every such block holds the same uniform index value (2-bit for BC1, 3-bit per channel +// for BC4 / BC5). A block with identical endpoints then stores no index payload: the header value is +// expanded to all 16 indices, instead of the full index payload. // // On a corrupted stream the decode runs to completion and the output is garbage, mirroring the // CPU version which only reports the failure after the whole strip was decoded. A valid stream @@ -400,7 +401,9 @@ kernel void bcp_decompress_bc1(const device uint8_t* input [[buffer(0)]], uint top_table_size = input[pos++] + 1; if (pos + 1 > length) return; - const bool flat_index_mode = input[pos++] != 0; // flat-index mode, right after the top-table size + const uint8_t flat_index_byte = input[pos++]; // flat-index mode + value, right after the top-table size + const bool flat_index_mode = (flat_index_byte & 1) != 0; + const uint8_t flat_index_value = (uint8_t)((flat_index_byte >> 1) & 3); pos = (pos + 3) & ~(uint64_t)3; // 4-byte align if (pos + 20 > length) @@ -552,14 +555,11 @@ kernel void bcp_decompress_bc1(const device uint8_t* input [[buffer(0)]], } // a flat block (identical endpoints, reconstructed exactly like the encoder sees the source) in - // mode 1 stores one 2-bit value, expanded to all 16 indices; otherwise the full reference + - // mask + differences + // mode 1 stores no index payload: the header value is expanded to all 16 indices; otherwise + // the full reference + mask + differences uint32_t indices; if ((current_color0 == current_color1) && flat_index_mode) - { - uint8_t value = bs_read_bits(&compressed_stream, 2); - indices = (uint32_t)value * 0x11111111u; - } + indices = (uint32_t)flat_index_value * 0x11111111u; else { uint8_t reference = decode_symbol(&compressed_stream, &reference_model); @@ -661,13 +661,18 @@ void bc45_decode_strip(const device uint8_t* input, seed[c][e] = input[pos++]; } - // flat-index mode: one byte per channel, after the seeds, before the 2-align + // flat-index mode + value: one byte per channel (mode | value << 1), after the seeds, before the 2-align if (pos + NumChannels > length) return; bool flat_mode[NumChannels]; + uint8_t flat_value[NumChannels]; for (uint c = 0; c < NumChannels; ++c) - flat_mode[c] = input[pos++] != 0; + { + uint8_t flat_byte = input[pos++]; + flat_mode[c] = (flat_byte & 1) != 0; + flat_value[c] = (flat_byte >> 1) & 7; + } //------------------------------------------------------------------------------------------------------------------------- // strip offsets (deltas in dword units), then the strip bitstream of this thread @@ -751,12 +756,11 @@ void bc45_decode_strip(const device uint8_t* input, // index payload: the first index in raw 3 bits, then each index xored with its // intra-block prediction, contextual on the 8 x 8 [left][top] matrix // a flat block (identical endpoints, reconstructed exactly like the encoder sees the - // source) in mode 1 stores one 3-bit value, expanded to all 16 indices + // source) in mode 1 stores no index payload: the header value is expanded to all 16 if ((e[0] == e[1]) && flat_mode[c]) { - uint8_t value = bs_read_bits(&compressed_stream, 3); for (uint i = 0; i < 16; ++i) - idx[i] = value; + idx[i] = flat_value[c]; } else { diff --git a/src/bc_packed.c b/src/bc_packed.c index 800fcab..d26f719 100644 --- a/src/bc_packed.c +++ b/src/bc_packed.c @@ -540,14 +540,16 @@ struct bcp_context // matching blocks) pay no flag overhead. see the per-strip dictionary enable in init_static_models uint64_t strip_dict_enable; - // flat-index mode of the whole stream: 1 → a block whose two endpoints are identical stores a single - // 2-bit index value, expanded to all 16 indices, instead of the reference + mask + differences; - // 0 → every block stores its indices as usual. decided once in the first pass: 1 only if at least - // one block has identical endpoints and every such block has a uniform index word (v * 0x11111111), - // else 0 (an encoder that leaves garbage indices on a flat block forces mode 0). the flag is written - // in the stream header's alignment padding, absorbed except in the worst parity case (a few bytes, - // covered by packed_bc1_maxsize) + // flat-index mode of the whole stream: 1 → a block whose two endpoints are identical stores no + // index payload; the decoder expands the 2-bit value from the stream header to all 16 indices, + // instead of the reference + mask + differences; 0 → every block stores its indices as usual. + // decided once in the first pass: 1 only if at least one block has identical endpoints and every + // such block holds the same uniform index word, else 0 (an encoder that leaves garbage or + // differing indices on flat blocks forces mode 0, keeping the roundtrip byte-exact). the mode and + // the value share one header byte (mode | value << 1) written in the stream header's alignment + // padding, absorbed except in the worst parity case (a few bytes, covered by packed_bc1_maxsize) uint8_t bc1_index_mode; + uint8_t bc1_index_value; // 2-bit value of the shared index word, expanded to all 16 indices, mode 1 only // bc4/bc5: per-channel static models — the 2 endpoint models (256 symbols) and the 64 // index models (8 symbols each, 8 x 8 [left][top] matrix); each retains the full 516-byte @@ -556,12 +558,14 @@ struct bcp_context le_model bc4_index_model[BCP_MAX_CHANNELS][BC4_INDEX_CONTEXTS]; uint8_t bc4_strip_seed[BCP_MAX_CHANNELS][2]; // [channel][endpoint] - // per-channel flat-index mode: 1 → a block whose two endpoints are identical stores a single 3-bit - // index value, expanded to all 16 indices, instead of the 15 contextual symbols; 0 → the indices - // are stored as usual. decided per channel in the first pass, same rule as the BC1 mode. one byte - // per channel in the stream header after the seeds, in the 2-byte alignment padding (worst case a - // couple bytes, covered by packed_bc4_maxsize / packed_bc5_maxsize) + // per-channel flat-index mode: 1 → a block whose two endpoints are identical stores no index + // payload; the decoder expands the 3-bit value from the stream header to all 16 indices, instead + // of the 15 contextual symbols; 0 → the indices are stored as usual. decided per channel in the + // first pass, same rule as the BC1 mode. the mode and the value share one header byte per channel + // (mode | value << 1) after the seeds, in the 2-byte alignment padding (worst case a couple bytes, + // covered by packed_bc4_maxsize / packed_bc5_maxsize) uint8_t bc4_index_mode[BCP_MAX_CHANNELS]; // [channel] + uint8_t bc4_index_value[BCP_MAX_CHANNELS]; // [channel] 3-bit value, expanded to all 16 indices, mode 1 only }; typedef struct bc1_block @@ -1268,11 +1272,14 @@ void init_static_models(bcp_context* ctx, const void* input, uint32_t width_bloc // symbols (the dictionary does not touch the indices). the color models fit on this pass price // the matched blocks in the per-strip dictionary decision below, the final color models are // refit on the deltas that are actually emitted after the decision - // the flat-index mode is decided here too: mode 1 (a block with identical endpoints stores one - // 2-bit index) only if at least one block has identical endpoints and every such block has a - // uniform index word, else mode 0 (garbage indices on a flat block force the full encoding) + // the flat-index mode is decided here too: mode 1 (a block with identical endpoints stores no + // index payload, the decoder expands the header value) only if at least one block has identical + // endpoints and every such block holds the same uniform index word, else mode 0 (garbage or + // differing indices on a flat block force the full encoding) bool flat_block = false; bool non_uniform_flat = false; + bool divergent_flat = false; + uint8_t flat_value = 0; for(uint32_t strip_index=0; strip_indexcolor[0] == current->color[1]) { - flat_block = true; const uint8_t value = (uint8_t)(current->indices & 3); if (current->indices != (uint32_t)value * 0x11111111u) non_uniform_flat = true; + if (!flat_block) + flat_value = value; + else if (value != flat_value) + divergent_flat = true; + flat_block = true; } uint8_t reference = nearest32(ctx->top_table, ctx->top_table_size, current->indices) & 0xff; @@ -1327,7 +1338,16 @@ void init_static_models(bcp_context* ctx, const void* input, uint32_t width_bloc } } - ctx->bc1_index_mode = (flat_block && !non_uniform_flat) ? 1 : 0; + if (flat_block && !non_uniform_flat && !divergent_flat) + { + ctx->bc1_index_mode = 1; + ctx->bc1_index_value = flat_value; + } + else + { + ctx->bc1_index_mode = 0; + ctx->bc1_index_value = 0; + } init_color_models(ctx, &hist); @@ -1540,9 +1560,9 @@ typedef struct bc4_hists // first block of each tile in chain order (as BC1), then one histogram pass with the exact // tile predictors of the encoder. there is no per-tile mode decision, so a single fit is final. // the per-channel flat-index mode is also decided in the histogram pass: mode 1 (a block whose -// two endpoints are identical stores one 3-bit index) only if at least one such block exists and -// every such block has uniform indices, else mode 0 (garbage indices on a flat block force the -// full 15-symbol encoding) +// two endpoints are identical stores no index payload, the decoder expands the header value) only if +// at least one such block exists and every such block holds the same uniform 3-bit value, else mode +// 0 (garbage or differing indices on a flat block force the full 15-symbol encoding) static void bc4_init_models(bcp_context* ctx, const void* image, uint32_t width_blocks, uint32_t height_blocks, uint32_t num_strips, uint32_t num_channels, uint32_t block_stride) { bc4_hists hist[BCP_MAX_CHANNELS] = { 0 }; @@ -1550,6 +1570,8 @@ static void bc4_init_models(bcp_context* ctx, const void* image, uint32_t width_ uint8_t prev_e[BCP_MAX_CHANNELS][2], top_e[BCP_MAX_CHANNELS][2], top_prev_e[BCP_MAX_CHANNELS][2]; bool flat_block[BCP_MAX_CHANNELS] = { false, false }; bool non_uniform_flat[BCP_MAX_CHANNELS] = { false, false }; + bool divergent_flat[BCP_MAX_CHANNELS] = { false, false }; + uint8_t flat_value[BCP_MAX_CHANNELS] = { 0, 0 }; // pass 1: tile seeds, the average of the first block of each tile in chain order for(uint32_t strip_index=0; strip_indexendpoint[0] == b->endpoint[1]) { - flat_block[c] = true; const uint8_t first = idx[0]; + if (!flat_block[c]) + flat_value[c] = first; + else if (first != flat_value[c]) + divergent_flat[c] = true; + for(uint32_t i=0; i<16; ++i) { if (idx[i] != first) @@ -1632,6 +1658,8 @@ static void bc4_init_models(bcp_context* ctx, const void* image, uint32_t width_ break; } } + + flat_block[c] = true; } for(uint32_t i=1; i<16; ++i) @@ -1657,7 +1685,18 @@ static void bc4_init_models(bcp_context* ctx, const void* image, uint32_t width_ } for(uint32_t c=0; cbc4_index_mode[c] = (flat_block[c] && !non_uniform_flat[c]) ? 1 : 0; + { + if (flat_block[c] && !non_uniform_flat[c] && !divergent_flat[c]) + { + ctx->bc4_index_mode[c] = 1; + ctx->bc4_index_value[c] = flat_value[c]; + } + else + { + ctx->bc4_index_mode[c] = 0; + ctx->bc4_index_value[c] = 0; + } + } for(uint32_t c=0; cbc4_strip_seed[c][e]; } - // flat-index mode, one byte per channel, written right after the seeds. the model region is always - // even-sized, so this byte sits on an odd stream position and the 2-align below can never land on - // it and zero it, keeping the encoder and the decoder in agreement + // flat-index mode + value, one byte per channel (mode | value << 1), written right after the + // seeds. the model region is always even-sized, so these bytes sit on an odd stream position and + // the 2-align below can never land on them and zero them, keeping the encoder and the decoder in + // agreement if (stream.pos + num_channels > stream.length) return 0; for(uint32_t c=0; cbc4_index_mode[c]; + stream.buffer[stream.pos++] = (uint8_t)(ctx->bc4_index_mode[c] | (ctx->bc4_index_value[c] << 1)); stream_align(&stream, sizeof(uint16_t)); size_t strips_offset_array_size = sizeof(uint16_t) * (*num_strips - 1); @@ -1788,12 +1828,11 @@ static size_t bcp_compress_bc45(bcp_context* ctx, const void* image, uint32_t wi // intra-block prediction, contextual on the 8 x 8 [left][top] matrix bc4_unpack_indices(b, idx); - // a flat block (identical endpoints) in mode 1 stores one 3-bit value, expanded to - // all 16 indices on decode; otherwise the full 15 contextual symbols + // a flat block (identical endpoints) in mode 1 stores no index payload: the + // decoder expands the header value to all 16 indices; otherwise the full + // 15 contextual symbols const bool flat = (b->endpoint[0] == b->endpoint[1]); - if (flat && ctx->bc4_index_mode[c]) - le_write_bits(&compressed_stream, idx[0], 3); - else + if (!(flat && ctx->bc4_index_mode[c])) { le_write_bits(&compressed_stream, idx[0], 3); @@ -1869,7 +1908,11 @@ static bool bcp_decompress_bc45(bcp_context* ctx, const void* input, size_t inpu if (stream.pos + num_channels > stream.length) return false; for(uint32_t c=0; cbc4_index_mode[c] = stream.buffer[stream.pos++]; // flat-index mode, after the seeds (see bc4_index_mode) + { + const uint8_t flat_byte = stream.buffer[stream.pos++]; // flat-index mode + value, after the seeds (see bc4_index_mode) + ctx->bc4_index_mode[c] = flat_byte & 1; + ctx->bc4_index_value[c] = (flat_byte >> 1) & 7; + } read_stream_align(&stream, sizeof(uint16_t)); size_t strips_offset_array_size = sizeof(uint16_t) * (num_strips - 1); @@ -1939,13 +1982,12 @@ static bool bcp_decompress_bc45(bcp_context* ctx, const void* input, size_t inpu // index payload: the first index in raw 3 bits, then each index xored with its // intra-block prediction, contextual on the 8 x 8 [left][top] matrix // a flat block (identical endpoints, reconstructed exactly like the encoder sees the - // source) in mode 1 stores one 3-bit value, expanded to all 16 indices + // source) in mode 1 stores no index payload: the header value is expanded to all 16 const bool flat = (b->endpoint[0] == b->endpoint[1]); if (flat && ctx->bc4_index_mode[c]) { - const uint8_t value = le_read_bits(&compressed_stream, 3); for(uint32_t i=0; i<16; ++i) - idx[i] = value; + idx[i] = ctx->bc4_index_value[c]; } else { @@ -2048,7 +2090,7 @@ static size_t bcp_compress_bc1_impl(bcp_context* ctx, const void* bc1_image, uin return 0; stream.buffer[stream.pos++] = (uint8_t) (ctx->top_table_size - 1); // there is no zero toptable, so minus 1 to fit in a uint8_t - stream.buffer[stream.pos++] = ctx->bc1_index_mode; // flat-index mode, written in the alignment padding (see bc1_index_mode) + stream.buffer[stream.pos++] = (uint8_t)(ctx->bc1_index_mode | (ctx->bc1_index_value << 1)); // flat-index mode + value, written in the alignment padding (see bc1_index_mode) stream_align(&stream, sizeof(uint32_t)); for(uint32_t j=0; j<4; ++j) @@ -2161,12 +2203,11 @@ static size_t bcp_compress_bc1_impl(bcp_context* ctx, const void* bc1_image, uin } } - // a flat block (identical endpoints) in mode 1 stores one 2-bit value, expanded to all - // 16 indices on decode; otherwise the full reference + mask + differences + // a flat block (identical endpoints) in mode 1 stores no index payload: the decoder + // expands the header value to all 16 indices; otherwise the full reference + mask + + // differences const bool flat = (current->color[0] == current->color[1]); - if (flat && ctx->bc1_index_mode) - le_write_bits(&compressed_stream, current->indices & 3, 2); - else + if (!(flat && ctx->bc1_index_mode)) { uint8_t reference = nearest32(ctx->top_table, ctx->top_table_size, current->indices) & 0xff; @@ -2246,7 +2287,11 @@ static bool bcp_decompress_bc1_impl(bcp_context* ctx, const void* input, size_t ctx->top_table_size = (uint32_t)stream.buffer[stream.pos++] + 1; if (stream.pos + 1 > stream.length) return false; - ctx->bc1_index_mode = stream.buffer[stream.pos++]; // flat-index mode, written right after the top-table size (see bc1_index_mode) + { + const uint8_t flat_byte = stream.buffer[stream.pos++]; // flat-index mode + value, written right after the top-table size (see bc1_index_mode) + ctx->bc1_index_mode = flat_byte & 1; + ctx->bc1_index_value = (flat_byte >> 1) & 3; + } read_stream_align(&stream, sizeof(uint32_t)); if (stream.pos + sizeof(uint32_t) + 8 + 8 > stream.length) @@ -2365,14 +2410,11 @@ static bool bcp_decompress_bc1_impl(bcp_context* ctx, const void* input, size_t } // a flat block (identical endpoints, reconstructed exactly like the encoder sees the source) in - // mode 1 stores one 2-bit value, expanded to all 16 indices; otherwise the full reference + - // mask + differences + // mode 1 stores no index payload: the header value is expanded to all 16 indices; otherwise + // the full reference + mask + differences const bool flat = (current->color[0] == current->color[1]); if (flat && ctx->bc1_index_mode) - { - const uint8_t value = le_read_bits(&compressed_stream, 2); - current->indices = (uint32_t)value * 0x11111111u; - } + current->indices = (uint32_t)ctx->bc1_index_value * 0x11111111u; else { uint32_t reference = le_decode_symbol(&compressed_stream, &ctx->table_reference_model); diff --git a/src/bc_packed.h b/src/bc_packed.h index 8792e55..ce90da4 100644 --- a/src/bc_packed.h +++ b/src/bc_packed.h @@ -14,7 +14,7 @@ extern "C" { // version #define BC_PACKED_MAJOR 1 #define BC_PACKED_MINOR 0 -#define BC_PACKED_PATCH 0 +#define BC_PACKED_PATCH 1 //---------------------------------------------------------------------------------------------------------------------------- diff --git a/tests/synthetic_tests.c b/tests/synthetic_tests.c index 347e989..9f5ba33 100644 --- a/tests/synthetic_tests.c +++ b/tests/synthetic_tests.c @@ -190,6 +190,113 @@ TEST identical_colors(void) return roundtrip(image, TEST_IMAGE_WIDTH, TEST_IMAGE_HEIGHT); } +//----------------------------------------------------------------------------------------------------------------------------- +// Identical endpoints all sharing the same non-zero uniform index word: the whole-image +// flat-index value (mode 1, value 2) stored in the stream header +TEST identical_colors_uniform_value(void) +{ + bc1_block* image = malloc(TEST_IMAGE_SIZE * sizeof(bc1_block)); + if (!image) + return GREATEST_TEST_RES_FAIL; + + for(uint32_t i=0; iindices[j] = (uint8_t)(word >> (8 * j)); +} + +//----------------------------------------------------------------------------------------------------------------------------- +// one solid block value per channel with a near-uniform 0x33 word (6, 3, 1 repeating as 3-bit +// values): the non-uniform word forces the full 15-symbol index encoding (mode 0), as bc45_flat TEST bc45_flat_uniform(void) { bc4_block image[TEST_BC45_BLOCKS]; @@ -684,6 +802,73 @@ TEST bc45_flat_uniform(void) return bc45_both(image, image16, TEST_BC45_WIDTH, TEST_BC45_HEIGHT); } +//----------------------------------------------------------------------------------------------------------------------------- +// one solid block value per channel with a TRULY uniform index word (all 16 indices the same): +// selects the flat-index mode 1, the value stored once per channel in the stream header +TEST bc45_flat_uniform_word(void) +{ + bc4_block* image = malloc(TEST_BC45_BLOCKS * sizeof(bc4_block)); + bc4_block* image16 = malloc(TEST_BC45_BLOCKS * 2 * sizeof(bc4_block)); // 16-byte BC5 block: the red bc4_block followed by the green one + if (!image || !image16) + { + free(image); + free(image16); + return GREATEST_TEST_RES_FAIL; + } + + for(uint32_t i=0; i