Repository navigation
Give every kept scan a version from each model the site offers - #352
Merged
Merged
Conversation
The idle-time second-opinion process ran each kept scan through ePAI, Atlas-Net and LesionSegmenter only. Add SuPreM, R-Super-Pancreas-Kidney and R-Super-Merlin, so a scan ends up with a version from every model in the picker. All three write combined_labels.nii.gz in the viewer's label scheme, so the existing saver works unchanged. Their weights are single files, not nnU-Net folders, so provenance gets a single-file rule (checkpoint and class-list hashes, fixed settings). The R-Super releases have no label map, so label_support says known: false instead of guessing. A test now fails if the default list and the model picker drift apart.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
When a scan is kept in
UserData, an idle-time process (scripts/user_dataset_second_opinions.py, running on bdmap1 since Oct 16) runs it through the other models whenever the site is idle and a GPU worker is free, and saves each result beside the original. It only did this for ePAI, Atlas-Net and LesionSegmenter. The site now offers three more models.What
The default list is now every model in the picker: ePAI, Atlas-Net, LesionSegmenter, SuPreM, R-Super-Pancreas-Kidney, R-Super-Merlin. Everything else about the process is unchanged (only while the site is idle, only on a GPU worker, cancelled the moment a user job appears, daily cap, back-off, one atomic folder per version).
combined_labels.nii.gzin the viewer's label scheme (checked by running each on a test scan through the production run path), so the saver needs no change.services/model_provenance.pygets a single-file rule: the checkpoint and the class list are hashed intoversion.json, with the settings the runners fix (SuPreM: U-Net backbone; R-Super: whether lesions are masked by the model's own organ masks).MODEL_TO_VIEWER, solabel_supportsaysknown: falsefor them instead of guessing which labels they can produce..env.exampleand the script's help text updated.Rollout
The running process holds the old list in memory, so after deploying it has to be restarted once (stop file, then start; documented in
deploy/USER_DATASET_SECOND_OPINIONS.md). The backlog is 387 kept scans x up to 3 new models, paced by the existing cap of 100 runs per rolling 24 hours (about 12 days); raiseSECOND_OPINIONS_DAILY_CAPif the workers are known to be idle.Tests
test_user_dataset_models.py: the default list equals the models inUploadPage.tsx's picker (fails if a model is added to one and not the other, and with the list set back to three); every default model has a checkpoint rule and settings; a kept scan is owed a version from every other model. The loop tests now use an explicit three-model list so they test the loop, not the default.test_model_provenance.py: the single-file rules equal the runners' variables, defaults and fixed settings (drift guard); hashing the weights and class list; missing file and missing class list; label support for the R-Super releases. Fails when the SuPreM rule is removed.test_user_mri_dataset.py::test_daily_caps_apply_per_userfails in that combined run on main too (it passes alone); not touched here.