Skip to content

Document the model-server slot settings, warn when they drift apart, and show leases in /api/health - #361

Merged
aperson30 merged 1 commit into
mainfrom
chore/nninteractive-ops-and-visibility
Nov 13, 2026
Merged

aperson30 merged 1 commit into
mainfrom
chore/nninteractive-ops-and-visibility

Conversation

@aperson30

Copy link
Copy Markdown
Collaborator

Follow-up to #360. That PR makes a busy model server legible and frees the slot when a tab closes; this one covers the settings that caused the "busy" in the first place, and makes them visible.

Cause. The model server's --max-sessions and the backend's NNINTERACTIVE_MAX_SESSIONS have to match, and nothing documented that. Production ran the model server with 1 slot while the backend, and the code's own comments, assume 3 ("below the model-server's three-session cap"). "Refine with model" opens its own short-lived session next to a person's click session, so with one slot it was refused whenever anyone else, or that person's earlier click tool (up to 10 minutes idle), held the slot. In the model server's log, 9 of 24 session claims had been refused.

Changed in this PR

  • docs/operations-reference.md: how the slots work, what one person needs (2), how to run the model server as a systemd user service, and how to change the count safely. .env.example now says to keep the backend limit no larger than the server's.
  • nninteractive_predictor.py: log a warning, at most once a minute, when the model server refuses a lease while the backend holds fewer than its own limit (the drifted-settings case).
  • /api/health: new interactive: {leases, max, per_user} (this backend's own leases) so "busy" can be told apart from "broken". It still answers if the model client is unavailable.
  • Refine hint text: it can also trim the class's own voxels (the code does), and a stale test comment is corrected.

Already done on the lab host (not part of the diff): a new model server with 3 slots now runs as a systemd user service on port 1530, and the backend's .env was pointed at it with NNINTERACTIVE_MAX_SESSIONS=3 and NNINTERACTIVE_MAX_SESSIONS_PER_USER=2, then the backend was restarted at 0 running jobs. The old 1-slot server is still running unused as a fallback.

Checked

  • Against the new server: 3 sessions open at once, the 4th is refused, releasing frees slots; the backend's own session logic then behaves as designed (an idle session is reclaimed before a new reader is refused). Three simultaneous first predictions on a synthetic volume finished about 4 s apart, because GPU work is serialised.
  • Real Refine on a real full-resolution scan (case 1, 136 MB as float32) through the backend's own predictor: 1 to 4 s each. On masks I made too big or too small it moved them toward the dataset label (Dice, liver 0.92 to 0.97, spleen 0.85 to 0.96, left kidney 0.79 to 0.91, pancreas 0.76 to 0.83), but on masks that already matched the label it changed them by 3 to 9 points of Dice (liver 0.97, spleen 0.97, left kidney 0.91, pancreas 0.91). So Refine helps rough masks and can slightly move good ones. Dice against the dataset label is an imperfect yardstick.
  • 6 new Python tests (46 pass in the interactive and health files; the 2 failures in test_interactive_session_ownership.py are identical on unmodified main in my environment). Removing each change fails a test: no warning, warning with a full pool, no rate limit, no health field, unguarded health.
  • Typecheck, ESLint and the toolbar/flyout tests pass.

Not verified: a real Refine click in a signed-in browser, and whether the 3-slot setting is enough under real concurrent use (traffic looks low: about 15 successful session claims in 13 days of the model server's log).

…and show leases in /api/health

The model server's --max-sessions and the backend's NNINTERACTIVE_MAX_SESSIONS must
match, and nothing said so: production ran the server with 1 slot while the backend
(and the code's own comments) assumed 3, so Refine, which needs a second lease next
to a user's click session, was refused whenever anyone else, or the same person's
earlier click tool, held the only slot.

- docs: explain the slots, what each user needs, how to run the model server as a
  systemd user service, and how to change the count safely.
- backend: log a warning (at most once a minute) when the model server refuses a
  lease while the backend holds fewer than its own limit, which is exactly the
  mismatched-settings case.
- /api/health: report this backend's model leases (interactive.leases / max /
  per_user) so "busy" can be told apart from "broken". The health route still
  answers if the model client is unavailable.
- Refine hint: it can trim the class's own voxels as well as add unlabeled ones, and
  it never overwrites another structure; also fix a stale test comment that said the
  flyout no longer exists.
@aperson30
aperson30 merged commit b12e57e into main Nov 13, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant