Skip to content

Give every kept scan a version from each model the site offers - #352

Merged
aperson30 merged 1 commit into
mainfrom
feat/second-opinions-new-models
Nov 8, 2026
Merged

aperson30 merged 1 commit into
mainfrom
feat/second-opinions-new-models

Conversation

@aperson30

Copy link
Copy Markdown
Collaborator

Why

When a scan is kept in UserData, an idle-time process (scripts/user_dataset_second_opinions.py, running on bdmap1 since Oct 16) runs it through the other models whenever the site is idle and a GPU worker is free, and saves each result beside the original. It only did this for ePAI, Atlas-Net and LesionSegmenter. The site now offers three more models.

What

The default list is now every model in the picker: ePAI, Atlas-Net, LesionSegmenter, SuPreM, R-Super-Pancreas-Kidney, R-Super-Merlin. Everything else about the process is unchanged (only while the site is idle, only on a GPU worker, cancelled the moment a user job appears, daily cap, back-off, one atomic folder per version).

  • All three new models already write combined_labels.nii.gz in the viewer's label scheme (checked by running each on a test scan through the production run path), so the saver needs no change.
  • Their weights are single files, not nnU-Net folders. services/model_provenance.py gets a single-file rule: the checkpoint and the class list are hashed into version.json, with the settings the runners fix (SuPreM: U-Net backbone; R-Super: whether lesions are masked by the model's own organ masks).
  • The R-Super releases have no entry in MODEL_TO_VIEWER, so label_support says known: false for them instead of guessing which labels they can produce.
  • Docs, .env.example and the script's help text updated.

Rollout

The running process holds the old list in memory, so after deploying it has to be restarted once (stop file, then start; documented in deploy/USER_DATASET_SECOND_OPINIONS.md). The backlog is 387 kept scans x up to 3 new models, paced by the existing cap of 100 runs per rolling 24 hours (about 12 days); raise SECOND_OPINIONS_DAILY_CAP if the workers are known to be idle.

Tests

  • test_user_dataset_models.py: the default list equals the models in UploadPage.tsx's picker (fails if a model is added to one and not the other, and with the list set back to three); every default model has a checkpoint rule and settings; a kept scan is owed a version from every other model. The loop tests now use an explicit three-model list so they test the loop, not the default.
  • test_model_provenance.py: the single-file rules equal the runners' variables, defaults and fixed settings (drift guard); hashing the weights and class list; missing file and missing class list; label support for the R-Super releases. Fails when the SuPreM rule is removed.
  • 150 tests pass in the user-dataset set (137 on main). test_user_mri_dataset.py::test_daily_caps_apply_per_user fails in that combined run on main too (it passes alone); not touched here.

The idle-time second-opinion process ran each kept scan through ePAI,
Atlas-Net and LesionSegmenter only. Add SuPreM, R-Super-Pancreas-Kidney
and R-Super-Merlin, so a scan ends up with a version from every model in
the picker.

All three write combined_labels.nii.gz in the viewer's label scheme, so
the existing saver works unchanged. Their weights are single files, not
nnU-Net folders, so provenance gets a single-file rule (checkpoint and
class-list hashes, fixed settings). The R-Super releases have no label
map, so label_support says known: false instead of guessing.

A test now fails if the default list and the model picker drift apart.
@aperson30
aperson30 merged commit 2a59688 into main Nov 8, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant