LLM & Compression: genomic sequences

Progress report

This work was defended as an M.Sc. thesis at the Faculty of Informatics, Masaryk University, and published as a workshop paper at the GenBio workshop at ICML 2026: [OpenReview], [poster].

Interactive information-content map: https://genomeinfo.dyn.cloud.e-infra.cz/.

Tasks

1. Test existing genomic language models.
2. Determine their compression ratio on the human genome (or other organisms) and compare it with standard compression models (e.g., gzip, 7zip).
3. It could also be interesting to analyze how much of the compression is attributed to the tokenizer versus the language model itself.
4. Since, unlike natural language, the compressibility of different genomic regions will vary significantly, it might be interesting to create an "information content map" across the human genome.

• src/llm_and_compression/arithmetic_encoder/GPTProbabilityModel.py - LLM interface responsible for token-by-token language modelling designed to be plugged into Arithmetic encoder.
• src/llm_and_compression/arithmetic_encoder/coder.py - implementation of the encoding algorithm.

How to get started

1. Clone the repository.
2. Install uv package:

pip install uv

3. Install the package in editable mode:

uv pip install -e .

Example compression notebook

An example notebook demonstrating how to use the implemented framework for compressing genomic sequences using LLMs can be found at: src/notebooks/compression_demo.ipynb.