Repository navigation
Document the model-server slot settings, warn when they drift apart, and show leases in /api/health - #361
Merged
Conversation
…and show leases in /api/health The model server's --max-sessions and the backend's NNINTERACTIVE_MAX_SESSIONS must match, and nothing said so: production ran the server with 1 slot while the backend (and the code's own comments) assumed 3, so Refine, which needs a second lease next to a user's click session, was refused whenever anyone else, or the same person's earlier click tool, held the only slot. - docs: explain the slots, what each user needs, how to run the model server as a systemd user service, and how to change the count safely. - backend: log a warning (at most once a minute) when the model server refuses a lease while the backend holds fewer than its own limit, which is exactly the mismatched-settings case. - /api/health: report this backend's model leases (interactive.leases / max / per_user) so "busy" can be told apart from "broken". The health route still answers if the model client is unavailable. - Refine hint: it can trim the class's own voxels as well as add unlabeled ones, and it never overwrites another structure; also fix a stale test comment that said the flyout no longer exists.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #360. That PR makes a busy model server legible and frees the slot when a tab closes; this one covers the settings that caused the "busy" in the first place, and makes them visible.
Cause. The model server's
--max-sessionsand the backend'sNNINTERACTIVE_MAX_SESSIONShave to match, and nothing documented that. Production ran the model server with 1 slot while the backend, and the code's own comments, assume 3 ("below the model-server's three-session cap"). "Refine with model" opens its own short-lived session next to a person's click session, so with one slot it was refused whenever anyone else, or that person's earlier click tool (up to 10 minutes idle), held the slot. In the model server's log, 9 of 24 session claims had been refused.Changed in this PR
docs/operations-reference.md: how the slots work, what one person needs (2), how to run the model server as a systemd user service, and how to change the count safely..env.examplenow says to keep the backend limit no larger than the server's.nninteractive_predictor.py: log a warning, at most once a minute, when the model server refuses a lease while the backend holds fewer than its own limit (the drifted-settings case)./api/health: newinteractive: {leases, max, per_user}(this backend's own leases) so "busy" can be told apart from "broken". It still answers if the model client is unavailable.Already done on the lab host (not part of the diff): a new model server with 3 slots now runs as a systemd user service on port 1530, and the backend's
.envwas pointed at it withNNINTERACTIVE_MAX_SESSIONS=3andNNINTERACTIVE_MAX_SESSIONS_PER_USER=2, then the backend was restarted at 0 running jobs. The old 1-slot server is still running unused as a fallback.Checked
test_interactive_session_ownership.pyare identical on unmodified main in my environment). Removing each change fails a test: no warning, warning with a full pool, no rate limit, no health field, unguarded health.Not verified: a real Refine click in a signed-in browser, and whether the 3-slot setting is enough under real concurrent use (traffic looks low: about 15 successful session claims in 13 days of the model server's log).