RetFold: Retrieval-Calibrated Evaluation of Protein Backbone Novelty

This repository contains the code used to generate RetFold backbones and audit structural novelty metrics against CATH S40. It includes the generation, ProteinMPNN/AlphaFold3 validation, domain segmentation, Foldseek retrieval, normalization audits, leave-one-topology-out controls, natural full-chain controls, and manuscript plotting scripts.

The repository intentionally excludes generated structures, model weights, third-party software, database files, AlphaFold3 outputs, and other large artifacts. Paths to those resources are supplied through environment variables or local configuration files.

Repository layout

.
├── scripts/                         experiment and analysis entry points
├── config/                          database-manifest template
├── configs/                         path-configuration template
├── docs/                            detailed rerun protocol
├── Figures/                         manuscript figures tracked by the paper
├── retrieval_baselines_r_1_r_2_r_3.py
├── retrieval_optimization.py        RetFold retrieval/assembly implementation
└── aaai27_retfold*.tex              manuscript and checklist sources

Software requirements

Core Python dependencies are listed in requirements.txt. The complete pipeline additionally uses:

• Foldseek
• CATH v4.4 and the CATH S40 non-redundant domain set
• ProteinMPNN
• AlphaFold3
• Merizo
• Chainsaw, for the segmentation-robustness experiment

Install the lightweight analysis dependencies with:

python -m pip install -r requirements.txt

Model-specific environments should follow the corresponding upstream projects. AlphaFold3 weights and databases are not distributed by this repository.

Local configuration

Copy the public templates and edit only the local copies:

cp config/novelty_databases.example.tsv config/novelty_databases.tsv
cp configs/default_paths.example.json configs/default_paths.json

The local files are ignored by Git. Most shell entry points also accept overrides such as:

WORK_DIR=/path/to/AAAI27-RetFold
FOLDSEEK=/path/to/foldseek
CATH_DB=/path/to/foldseek_cath_s40
THREADS=32

Main reproduction workflows

RetFold generation

The staged rerun protocol is documented in docs/retfold_rerun_protocol.md. Its main driver is:

bash scripts/run_retfold_full_experiment.sh status
bash scripts/run_retfold_full_experiment.sh stage1
# Continue with the staged ProteinMPNN, AF3, Stage 2, Merizo, and Foldseek modes
# described in the protocol.

Uniform full-chain and domain retrieval

bash scripts/run_uniform_fullchain_retrieval.sh status
bash scripts/run_unconditional_domain_retrieval.sh status

Operational-novelty and normalization audit

bash scripts/run_operational_novelty.sh all
bash scripts/run_foldseek_normalization_audit.sh all

Domain-sized leave-one-topology-out calibration

THREADS=32 MAX_SEQS=2000 \
bash scripts/run_foldseek_loto_specificity.sh all

Length-matched natural full-chain calibration

This experiment samples 100 non-S40 natural chains in each of five length bins, then evaluates both a non-verbatim known-topology positive control and a topology-masked full-chain negative control from one Foldseek search.

THREADS=32 \
DOWNLOAD_WORKERS=16 \
MAX_SEQS=2000 \
PER_LENGTH=100 \
bash scripts/run_natural_fullchain_calibration.sh all

The main outputs are:

results/natural_fullchain_calibration/
├── natural_fullchain_manifest.csv
├── natural_fullchain_calibration_summary.csv
├── natural_fullchain_calibration_by_length.csv
├── natural_fullchain_calibration_by_domain_count.csv
└── natural_fullchain_score_quantiles.csv

Segmentation robustness

bash scripts/run_table10_all500_chainsaw.sh status
bash scripts/run_table10_all500_chainsaw.sh search

Task-matched BoltzGen and PXDesign evaluation

bash scripts/run_boltzgen_pxdesign_unconditional.sh all
bash scripts/run_unconditional_mpnn_af3.sh status
bash scripts/run_unconditional_pairwise_diversity.sh summary

Reproducibility conventions

• The all-generated cohort contains 100 structures at each target length (100, 200, 300, 400, and 500 residues) unless explicitly stated otherwise.
• Foldseek searches use sensitivity 9.5, exact TM-score recomputation, and explicit qTM, tTM, and alnTM fields.
• CATH topology is represented by the first three CATH levels (Class.Architecture.Topology).
• Raw Foldseek outputs are retained locally but are not committed because of their size.
• Generated outputs and manuscript numbers should be regenerated from the scripts rather than edited manually.

Data and code availability

The source code is intended for public release with the paper. Generated structures, per-query score tables, and large intermediate files should be deposited in an archival data repository and linked here when an accession or DOI is available.

Security

Do not commit API keys, credentials, private model weights, or machine-local configuration. Use environment variables or ignored local configuration files.