[freshyjmp]

Summary

Pivot the Cosmos3-Nano/Brev launchable from VANTAGE to MVBench in the maintained Cosmos Framework VLMEvalKit skill. This remains the draft Nano/Brev track superseding NVIDIA/cosmos#382; Cosmos3.5 Ultra/LaunchPad is separate and unchanged.

• Run public download, trusted preparation, native inference, local exact-match scoring, and results directly from the Jupyter notebook. Default to a clearly labeled fixed five-question Egocentric Navigation smoke; full 4,000-question / 20-task evaluation is explicit opt-in and never silently reduced.
• Pin OpenGVLab/MVBench to 230a2d4fac8900333c61754641c7a13e069ac9c6, verify annotation/archive hashes, safely extract current public archives including the official June-2024 overlay, and refuse incomplete selected media coverage. Never retrieve removed NTU media from historical revisions or mirrors.
• Reuse native MVBench task definitions, temporal bounds, sampled-image prompts, and exact-match evaluator. Use the reference 16-frame / 224-resolution recipe with explicit Cosmos image-encoding and parser differences; do not claim VideoChat2 paper parity.
• Retain pinned Nano revision e59a53c25979a090fa8706c9acc0c254a6e89b92 and the digest-locked vLLM image. Preserve private authenticated Jupyter, an unexposed model API, external writable data, separate cache/results mounts, and one API worker by default.
• Support offline CPU scoring of existing predictions without a GPU, endpoint, API judge, private S3, or external scoring submission. Enforce exact identities, trusted labels, fresh outputs, and explicit scope; retain native counts, per-task metrics, aggregation units, hashes, and generation/source provenance.
• Remove the former launchable-specific VANTAGE local-bundle integration; retain unrelated upstream benchmark adapters and recipes unchanged.

Verification

Runtime candidate b0861ce448a648b2eb59c41a9358980bbb5df31c; subsequent commit updates validation documentation only.

• 28 behavioral regressions passed inside the newly built native Linux x86-64 evaluator image. Fixtures are synthetic; tests cover coverage, identities, ground-truth tampering, malformed/failure answers, micro/macro aggregation, native frame/bounds/cache behavior, safe archive extraction, and artifact/state error boundaries.
• Actual JupyterLab Run All Cells completed all six code cells, including five valid Nano predictions with matching identities and native local scoring, on the existing single RTX PRO 6000 Blackwell Server Edition environment. This is a smoke subset, not a full benchmark result; correctness artifacts remain private.
• The unchanged saved real predictions were rescored in a separate CPU-only container with --network none and offline library settings. Correctness counts, per-task metrics, and original prediction hashes matched.
• Current public archives were SHA-verified and extracted. Actual full preparation found media for 3,800 / 4,000 rows and rejected the run before inference because all 200 Fine-grained Pose NTU clips were absent. No rows were dropped and no partial full score was produced.
• Outgoing publication audit: 16 code/documentation/synthetic-test files, zero committed notebook outputs/execution counts/attachments, no data/cache/result/media files, no internal deployment details, and zero findings from redacted gitleaks 8.30.1 scanning.

Data and publication boundary

Code only. No benchmark videos/images, downloaded annotations, canonical TSVs, prediction files, score reports, executed notebook outputs, caches, credentials, internal staging addresses, credential-forwarding tools, or private operational evidence are included. Nothing is merged. No LaunchPad deployment or customer-data ingestion agent is included.

Public annotation availability does not grant unrestricted media or commercial redistribution rights. Original source agreements and the dataset card's academic-research restriction still apply.

Limits / release gates

Keep this PR draft. The full 20-task run is blocked on authorized NTU RGB+D media acquisition through NTU/ROSE Lab. The current dataset card mentions 320 manual videos, but the exact pinned video/MVBench_videos_ntu.txt inventory and this release's Fine-grained Pose task require 200 RGB AVI files; use that pinned inventory, not a different benchmark subset.

Validation reused an existing warm Nano endpoint and a manually staged evaluator. No fresh-VM one-click cold launch, complete 4,000-question run, named published Nano reference-score reproduction, end-to-end BYOM result, commercial/distribution approval, or customer-data agent implementation is claimed. The old VANTAGE validation is historical evidence only and is not carried forward as MVBench proof.