Skip to content

Add cohort template building and export analysis levels - #212

Merged
akhanf merged 6 commits into
mainfrom
cohort-template
Oct 24, 2026
Merged

akhanf merged 6 commits into
mainfrom
cohort-template

Conversation

@akhanf

@akhanf akhanf commented Oct 24, 2026 •

Copy link
Copy Markdown
Member

Adds two analysis levels for building a study-specific SPIM template with FireANTs and preparing it for use as a --template in later runs. See docs/howto/cohort_template.md.

cohort_template

  • Runs any missing subject registrations to --template. Each stain is resampled to template space at --cohort-template-level (default --registration_level).
  • Builds one multivariate template from all stains (--cohort-template-stains). Every stain template therefore shares the same space.
    • Subjects can be pooled from other SPIMquant output dirs with --cohort-template-extra-inputs.
    • Uses multiple GPUs with --cohort-template-n-gpus.
  • Registers the original template's anat to the cohort template (greedy affine + deformable), to correct template drift.
    • Warps the original atlases, brain mask and anat into cohort space with that registration.
    • Writes a regqc report. reg_qc_report.py now takes its title fields from params when the rule has no matching wildcards.

cohort_template_export

  • Writes template_description.json, a buildinfo tarball, and tpl-{name}_spimquant.yml: a config file that adds and selects the template, so any run can use it with --configfile (external users). Making it a built-in --template choice is done by a separate PR.
  • Can upload these to a Zenodo draft deposition (--cohort-template-zenodo-deposition).

Environment

  • fireants (khanlab fork) and fireants-fused-ops 1.2.0 are dependencies of the gpu pixi feature, both from akhanf/khanlab. Run this level with pixi run -e gpu spimquant ....
  • The fused CUDA ops are packaged in Fix fused ops conda CI FireANTs#5.
  • pixi.lock is regenerated, which also upgrades the lock format from v6 to v7.

GPU memory and defaults

I tested on an 18-subject PI cohort (mouse_app_mapt_apoe, level 5, ABAv3 25 µm) on a 40 GB A100:

shape avg, 50 µm no shape avg, 50 µm no shape avg, 25 µm
Build time 7.7 min 7.4 min 49.5 min
Subject NCC to template (scored at 50 µm) 0.899 0.928 0.937
Mean drift from ABAv3 86 µm 92 µm 84 µm
  • Shape averaging: cohort_template.fireants.shape_avg now defaults to false. It adds a full-resolution registration per iteration and runs out of memory at 25 µm, even with the fused ops. Without it, every subject fit the template better, and drift from ABAv3 stayed about the same.
  • Resampling: the new --cohort-template-resample-mm resamples subject images to an isotropic voxel size before building. It is off by default. It's meant for cutting memory and run time, e.g. with a finer --cohort-template-level.
  • Allocator: torchrun runs with PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True. Without it, fragmentation caused out-of-memory errors after about 14 subjects.

Test plan

  • cohort_template completes end to end on the cohort above in all three configurations.
  • --cohort-template-resample-mm 0.05 gives 228×264×160 inputs at 50 µm, and the generated FireANTs config has shape_avg: false.
  • pixi install -e gpu installs fireants-fused-ops from the channel, and FireANTs uses it.
  • cohort_template_export (local, no upload) on noshape50; a participant run on one subject with the exported --configfile registers cleanly (NCC 0.86 to the template, regqc looks good).
  • Zenodo upload to a sandbox.zenodo.org draft deposition (--storage-zenodo-sandbox): noshape50 export files upload, with sizes matching the local files. In sandbox mode the exported config still points at zenodo.org URLs.

🤖 Generated with Claude Code

akhanf and others added 6 commits October 23, 2026 16:22
- cohort_template: build a multivariate (all stains) cohort template with
  FireANTs from subjects registered to --template, optionally pooling other
  SPIMquant output dirs; register the original template anat to the cohort
  template (greedy affine+deformable) and warp its atlases and mask into
  cohort template space, with a regqc report.
- cohort_template_export: write template_description.json, a SPIMquant
  templates: config snippet and buildinfo tarball, and optionally upload to a
  Zenodo deposition via snakemake-storage-plugin-zenodo.
- FireANTs runs in its own conda env (envs/fireants.yaml) since its PyPI pins
  conflict with SPIMquant's simpleitk.
- pixi.lock regenerated (also upgrades lock format v6 -> v7).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- shape_avg now defaults to false: it adds a full-resolution registration
  per iteration, OOMs at 25um on a 40GB GPU, and gave a worse subject fit
- --cohort-template-resample-mm resamples subject images to an isotropic
  voxel size before template building, to cut memory and run time (e.g.
  with a finer --cohort-template-level)
- set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True for torchrun, to
  avoid OOM from fragmentation after a few subjects
- add fireants-fused-ops (akhanf/khanlab) to the gpu environment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The exported tpl-{name}_spimquant.yml now selects the template (template:)
and points at the exported files' absolute paths (or the Zenodo URLs), so
it can be passed to any run with --configfile, without editing
snakebids.yml. Built-in --template choices are added by PR instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@akhanf
akhanf merged commit fa0a99c into main Oct 24, 2026
3 checks passed
@akhanf
akhanf deleted the cohort-template branch October 24, 2026 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant