Skip to content

Harden Servy V5 trained-model serving verification - #16

Draft
yashaskn8 wants to merge 3 commits into
mainfrom
feature/servy-v5-serving-verification-20261025
Draft

yashaskn8 wants to merge 3 commits into
mainfrom
feature/servy-v5-serving-verification-20261025

Conversation

@yashaskn8

Copy link
Copy Markdown
Owner

Verified base

  • Based on main 358c48cad5301dc7dbd68f7f7b4fcecb530f512e.
  • V5 already has a local SHA-pinned checkpoint loader and an opt-in Django reranker; private weights are intentionally not stored in Git.

Change

  • Make python manage.py verify_v5_reranker run the same actual rerank_candidates path as production, including raw logits, instead of a separate default-activation model.predict() probe.
  • Treat disabled model, unavailable checkpoint, legacy fallback, and non-finite scoring as smoke-verification failures.
  • Add synthetic tests for raw-logit identity activation and failure handling; document scope and deployment instructions.

Important limitations

  • Not a production promotion. The actual 2.27 GB V5 best_model checkpoint and its running environment were not accessible, and an independent holdout is not established.
  • No model artifacts, personal/company text, secrets, or customer data were committed.
  • Baseline MiniLM remains default, and V5 is opt-in.
  • Real-weight path: offline installer -> SERVY_RERANKER_*=... -> smoke command -> SERVY_TEST_REAL_V5=1 end-to-end check -> independent quality/latency/citation/safety evaluation.
  • This PR does not claim the trained model is installed or deployed.

Verification

  • Awaiting GitHub MySQL backend tests, synthetic V5 tests, frontend lint/build. No real-weight test is runnable in CI absent the private checkpoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant