skills: add cosmos3-qwen3-vl-robovqa-sft (Qwen3-VL-8B SFT on RoboVQA, one GPU, scored before and after) by sbabamohamad · Pull Request #343 · NVIDIA/cosmos-framework

sbabamohamad · 2026-10-29T22:35:47Z

What this adds

An agent skill that fine-tunes Qwen3-VL-8B-Instruct, a model from outside the Cosmos family, through the posttrain SFT path on RoboVQA (Apache-2.0), on one GPU, and scores it before and after with the same evaluator. Written to the cosmos3-skill-authoring rules.

.agents/skills/cosmos3-qwen3-vl-robovqa-sft/: SKILL.md (eight steps, one command each: setup, download, data, preflight, before, train, export, after; a "For an agent" block with the one-line request), scripts/run.sh (the step runner; --smoke uses six bundled clips and 2 updates; one line per stage on screen, full output in logs), scripts/prepare_robovqa.py (split, frames, verify), scripts/eval_robovqa.py (evaluator and scorer), references/measured-run.md, six bundled RoboVQA clips (1.8 MB, Git LFS), eval/eval.config with two smoke cases.
cosmos_framework/posttrain/configs/sft_qwen3_vl_8b_robovqa.py and its test: the recipe, kept in the package because the SFT entrypoint imports configs as modules.
Separate commit, checkpoint/conversion/hf.py: HF exports written in 5 GB shards. With Transformers 5's 50 GB default, an 8B export is a single file without model.safetensors.index.json and fails verify_index. Happy to split into its own PR.
docs/posttrain/guides/sft.md: one pointer to the skill.

Results (one GB300, two full runs at seed 0, framework `9a70e64`)

	exact match	token-F1	yes/no
zero-shot	0.548	0.676	0.918
after one epoch	0.632 / 0.620	0.809 / 0.798	0.953 / 0.946

Training 2 h 05–2 h 14; torch peak 143,566 MiB reserved.

Testing

All eight steps with --smoke on current main (7738e9e) on two hosts: a single-GB300 host and a GB300 + RTX PRO 6000 host (the script selects the largest GPU and builds apex for it). Split 4/0/2, frame check all_ok, preflight failed 0, 2 training updates to SFT DONE, export [materialize] OK, 750 tensors in 4 shards (the path that fails without the hf.py change), scoring before and after.
Agent test: fresh clone, Claude Code with Sonnet, the one-line request from SKILL.md; it found the skill, ran the eight --smoke steps in order unchanged, and reported each result line. No intervention.
setup from a fresh clone on both hosts; apex builds in 6 to 11 minutes.
The skill's split reproduces the measured runs' seed-0 split byte for byte; its scorer reproduces both runs' recorded scores exactly.
ruff 0.12.7 clean.
Not yet run: the full 125-update training on current main (in progress; result to follow here), the config test in the posttrain environment, x86_64 hosts, and the skill eval.

Eval cases

smoke-rows-and-preflight: split, frames, verify and preflight on the bundled clips.
smoke-zero-shot-score: the evaluator on the 2 bundled held-out clips.

Both fit one 80 GB GPU. Full training needs about 150 GB on one GPU and stops with a message on smaller cards. Smoke data: six clips from Tianli/robovqa @ eb58194 (Apache-2.0, LICENSE included).

… one GPU, scored before and after) A worked post-training example for a model outside the Cosmos family: full-parameter SFT of Qwen/Qwen3-VL-8B-Instruct on RoboVQA (Apache-2.0) through the posttrain SFT path on the i4-native seat, on one GPU, with the same evaluator before and after training. Written to the cosmos3-skill-authoring rules. - .agents/skills/cosmos3-qwen3-vl-robovqa-sft/: SKILL.md (steps 0 to 8; every block self-contained through scripts/env.sh), scripts/ prepare_robovqa.py (split, frames, verify), scripts/eval_robovqa.py (evaluator and scorer), references/measured-run.md, six bundled RoboVQA clips (1.8 MB, Git LFS) for the smoke variant, and eval/eval.config with two smoke cases that fit one 80 GB GPU. - cosmos_framework/posttrain/configs/sft_qwen3_vl_8b_robovqa.py: the recipe (one epoch of 125 updates of 64 clips, fusedadam lr 2e-5 with a cosine schedule, max_length 10240, one consolidated model save) and its test. - docs/posttrain/guides/sft.md: a pointer to the skill. Measured on one GB300 (aarch64, on the host, framework 9a70e64): exact match 0.548 zero-shot, 0.632 and 0.620 after one epoch in two runs; training 2 h 05 to 2 h 14, torch peak 143,566 MiB reserved. The skill's split reproduces the seed-0 split of those runs byte for byte, and its scorer reproduces their recorded scores exactly. Signed-off-by: sbabamohamad <sbabamohamad@nvidia.com>

…index Transformers 5 defaults save_pretrained's max_shard_size to 50 GB, so a model under 50 GB (Qwen3-VL-8B, 16.3 GiB) is written as one model.safetensors without model.safetensors.index.json, and the materializer's own verify_index refuses the export ("[materialize] FATAL ... model.safetensors.index.json missing"). Writing 5 GB shards, the Transformers 4 default, restores the index; the Qwen3-VL-8B export then verifies with 750 tensors in 4 shards. Signed-off-by: sbabamohamad <sbabamohamad@nvidia.com>

scripts/run.sh runs each step of the example as one command (setup, download, data, preflight, before, train, export, after; --smoke for the six bundled clips and 2 updates), so SKILL.md reads as eight one-line steps with their expected output. scripts/env.sh is folded into run.sh. Signed-off-by: sbabamohamad <sbabamohamad@nvidia.com>

The apex build printed nothing for about ten minutes; setup now says how long it takes and streams one line per compiled source file, and fails with a clear message when the build does not produce amp_C. The model download and the training launch also announce what to expect before their quiet stretches. Signed-off-by: sbabamohamad <sbabamohamad@nvidia.com>

Building apex through pip hid the compiler for the whole ten minutes (the progress filter matched nothing). setup now builds the wheel with setup.py directly, streams the compiler output and saves it to logs/apex-build.log, caps the parallel jobs at 32, and fails with the log's path when no wheel comes out. Signed-off-by: sbabamohamad <sbabamohamad@nvidia.com>

Every step of run.sh now prints one line per stage (the label, then "done" with the elapsed time) and a one-line result, and writes each stage's full output to artifacts/qwen3-vl-robovqa/logs/<stage>.log. A failing stage prints FAILED, the last 30 lines of its log and the log's path, and the step stops. Training shows only the step lines and the end marker; the framework's full log goes to the file. Signed-off-by: sbabamohamad <sbabamohamad@nvidia.com>

Two first-run failures seen on a two-GPU host without git-lfs: - A clone made without git-lfs holds pointer files for the six bundled clips, and the data step then fails inside the video decoder. The download step now checks for pointers, runs git lfs pull for the assets, and says what to install when git-lfs is missing. - nvidia-smi listed a 97 GB card first, so the training step refused. The script now selects the GPU with the most memory for every GPU step unless CUDA_VISIBLE_DEVICES is already set, and names the GPU it checked when it refuses. Also shortens the stage labels so they fit their column. Signed-off-by: sbabamohamad <sbabamohamad@nvidia.com>

setup treated any importable apex as usable. A build made for another GPU (here: compiled for the 97 GB card of a two-GPU host, then trained on the GB300) imports fine and fails at the first optimizer step with "no kernel image is available for execution on the device". setup now runs one fused kernel on the selected GPU and rebuilds apex when that fails; build_apex checks the same thing after installing. Signed-off-by: sbabamohamad <sbabamohamad@nvidia.com>

SKILL.md opens with the one-line request an agent runs the skill from and the rules it follows (order, expected lines, stop at FAILED), and says how to bring another dataset (SFT_DATA_ROOT in the framework's JSONL format, skip the data step) or another Qwen3-VL size (copy the recipe, change the model, pass it as CONFIG), and that other model families are outside the SFT path. run.sh reads CONFIG, SFT_MODEL_PATH and SFT_DATA_ROOT from the environment when set. Signed-off-by: sbabamohamad <sbabamohamad@nvidia.com>

…er result aside The framework refuses to start training at step 0 in a run directory that already holds a checkpoint, and the materializer refuses an existing output directory; a second run of the skill in the same checkout therefore failed at train (found by an agent running the smoke variant in a reused clone). Both steps now move the earlier result to a timestamped name and say so. Signed-off-by: sbabamohamad <sbabamohamad@nvidia.com>

christinayyw · 2026-10-30T02:33:57Z

Skill eval — affbdac · manual run (CI-equivalent)

8 / 8 cells done · 8 passed · 0 failed · 0 no report · 0 errors · 8 passed after self-correction · started 2026-10-29 23:23 UTC · updated 2026-10-30 00:29 UTC

`cosmos3-qwen3-vl-robovqa-sft` · dashboard overview

Case	claude	codex	cursor	opencode
`smoke-rows-and-preflight`	✅ PASS ↻ · report	✅ PASS ↻ · report	✅ PASS ↻ · report	✅ PASS ↻ · report
`smoke-zero-shot-score`	✅ PASS ↻ · report	✅ PASS ↻ · report	✅ PASS ↻ · report	✅ PASS ↻ · report

↻ Agent self-correction needed in all 8 cells. The first attempt of the documented workflow stopped in run.sh setup at the host check: scripts/run.sh:85 reads the CUDA version from the nvidia-smi banner with CUDA Version:, and this driver (615.71.10) prints CUDA UMD Version: 13.4, so the parsed version was empty and the check rejected the host. Each agent widened that regex and every later step then ran unchanged and matched its documented expected output. Each report's "Self-Correction Needed" line and "Fix Suggestions" section has the per-cell detail.

Legend: ✅ PASS · ❌ FAIL · ⚠️ PARTIAL · ❔ NO-REPORT (the agent finished without writing a report) · ⛔ ERROR (model gateway / agent / node error, see the report) · ↻ = passed after self-correction (the first attempt of the documented workflow failed; the agent fixed it and re-ran)

Run details: this PR comes from a fork, which the self-hosted eval workflow does not run, so the same matrix was run by hand on the eval runners with the CI orchestrator and image (cosmos-skill-eval:v20261102, evaluator 0.3.4), one backend per host, 1× H100 80GB (x86_64, driver 615.71.10). Static tier: 95/100.

✅ done — 8/8 passed (0 failed, 0 no report, 0 errors; 8 after self-correction)

lfengad

Thanks, this is a nice end-to-end recipe. Every command, module, flag and uv group the skill tells users to run exists at this head, and I found no internal references.

Needed before merge

The required pre-commit check fails on rumdl MD060 table alignment: 51 issues across SKILL.md and references/measured-run.md. rumdl check --fix (v0.2.78, the pinned hook) fixes all of them. Because pre-commit fails, the GPU workflow skipped unittest, so test_config_sft_qwen3_vl_8b_robovqa.py has not run in CI yet.
The hf.py fix will be reverted by the next posttrain sync (inline). The same applies to docs/posttrain/guides/sft.md.

Needed before this is quoted externally (11/3)

references/measured-run.md:6 says the numbers came from "an earlier version" of run.sh, and no scores or predictions file is committed. Please re-run with the current script, or commit the scores and predictions, so the 0.548 → 0.632 / 0.620 figures are reproducible from this PR.
"One GPU" means a GPU with at least 150,000 MiB free (run.sh:276), which rules out H100 and H200. Only GB300/aarch64 was verified. The table in SKILL.md already says this, but external copy should keep the qualifier.
Licensing (inline on SKILL.md:97). Please confirm it, and also state the Qwen3-VL-8B-Instruct license.

Nits are inline. The skill is also not registered the way its siblings are: it is missing from the skills table in README.md and from AGENTS.md, and has no .claude/skills/ symlink like cosmos-automl and cosmos-framework-docs. These nits can follow in a later PR.

…` banner; keep previous stage logs `check_host` read the CUDA version from the nvidia-smi banner with a regex that only matched `CUDA Version:`. R615 drivers print `CUDA UMD Version: 13.4` (the old label is deprecated), so the parse came back empty and `setup` stopped at its first stage with "the driver supports CUDA ;" on a supported CUDA 13.4 host. In the skill eval of PR NVIDIA#343 (group ci-pr343-affbdac, 4 agents x 2 cases on x86_64 H100, driver 615.71.10) this was the first-attempt failure of all 8 cells; every agent patched the regex and then passed. - accept `CUDA Version:` and `CUDA UMD Version:`; when neither parses, fail naming the banner line instead of reporting an empty version - `stage()` keeps the previous attempt's log as `<name>.log.prev` instead of overwriting it on a re-run - the `setup` log hint names the smoke variant's files (`apex-build-smoke.log`) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Christina Wang <christwang@nvidia.com>

…log names, banner troubleshooting - step 1 tells agents that `setup` is one 12-minute command: run it as one long-timeout call or detached like `train`, wait for its last lines, and re-run the same command if a shell tool stops it early (finished environments and the apex build are kept). Two of the eight eval cells lost a `setup` run to a 10-minute tool cap and recovered by re-running. - the host check installs missing tools with apt-get when run as root, otherwise prints the command and stops - the compiler log is `apex-build-smoke.log` with `--smoke`; a re-run keeps the previous log as `.log.prev` - "Things not obvious": the `could not read the CUDA version from the nvidia-smi banner` message Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: Christina Wang <christwang@nvidia.com>

The repository's rumdl configuration requires aligned table columns; SKILL.md and references/measured-run.md failed the pre-commit lint on 51 table lines. Formatting only (rumdl check --fix with .config/rumdl.toml); no text changed. Signed-off-by: sbabamohamad <sbabamohamad@nvidia.com>

) ## Summary **Stacked on #343:** based on #343's head `d94b180` (mirrored here as branch `skills/qwen3-vl-robovqa-sft-pr343-base` so the diff shows only the fixes). Skill-only: `scripts/run.sh` and `SKILL.md` of `cosmos3-qwen3-vl-robovqa-sft`; no framework files, no `eval/eval.config` change, so before and after are measured on the same cases, assertions and checks. Once reviewed, the two commits go onto #343's branch (maintainer push) and this PR closes. Follow-up to the skill eval of #343 ([comment](#343 (comment)), [dashboard group ci-pr343-affbdac](http://internal-cosmos-skills-eval.nvidia.com/skills/cosmos3-qwen3-vl-robovqa-sft/groups/ci-pr343-affbdac/)): 8/8 cells PASS, but 0/8 on the first attempt. Every first attempt failed on the same line, and the agents' remaining friction points are the LV2 suggestions of the eval reports. ### Fixes Where = `file:lines` at `1ad7352ea` (permalinks; `SKILL.md` = `.agents/skills/cosmos3-qwen3-vl-robovqa-sft/SKILL.md`, `run.sh` = `scripts/run.sh`). Problem → fix, in priority order. | ID | Pri | Problem (eval evidence) | Fix applied | Where | |---|---|---|---|---| | F1 | P0 | `check_host` parsed the nvidia-smi banner for `CUDA Version:` only; R615 drivers print `CUDA UMD Version: 13.4` (old label deprecated), so the version came back empty and `setup` stopped at stage 1 with "the driver supports CUDA ;" on a supported host. First-attempt failure of all 8 cells (LV1 in every report); each agent patched the regex and then passed. | Accept both labels; when neither parses, fail naming the banner line instead of a blank version | run.sh [86–95](https://github.com/NVIDIA/cosmos-framework/blob/1ad7352ea/.agents/skills/cosmos3-qwen3-vl-robovqa-sft/scripts/run.sh#L86-L95) | | F2 | P1 | `setup` is one 12-minute command; Claude Code's 10-minute shell cap killed it once in each of its two cells (re-run recovered, apex reused). SKILL.md told agents to detach only `train` (LV2, claude ×2). | Step 1 tells agents to run `setup` as one long-timeout call or detached like `train`, wait for its last lines, and re-run the same command if a tool stops it early (environments and apex build are kept) | SKILL.md [116–119](https://github.com/NVIDIA/cosmos-framework/blob/1ad7352ea/.agents/skills/cosmos3-qwen3-vl-robovqa-sft/SKILL.md#L116-L119) | | F3 | P2 | A re-run of a stage overwrote the previous attempt's log, losing the failure evidence (LV2, claude rows) | `stage()` keeps the previous log as `<name>.log.prev`; documented in "Verify the output" | run.sh [70](https://github.com/NVIDIA/cosmos-framework/blob/1ad7352ea/.agents/skills/cosmos3-qwen3-vl-robovqa-sft/scripts/run.sh#L70) · SKILL.md [199](https://github.com/NVIDIA/cosmos-framework/blob/1ad7352ea/.agents/skills/cosmos3-qwen3-vl-robovqa-sft/SKILL.md#L199) | | F4 | P2 | `setup` ended with "logs: … apex-build.log" but the smoke variant writes `apex-build-smoke.log` (LV2, claude zero-shot) | Hint and step 1 name the `$TAG` variant | run.sh [141](https://github.com/NVIDIA/cosmos-framework/blob/1ad7352ea/.agents/skills/cosmos3-qwen3-vl-robovqa-sft/scripts/run.sh#L141) · SKILL.md [114](https://github.com/NVIDIA/cosmos-framework/blob/1ad7352ea/.agents/skills/cosmos3-qwen3-vl-robovqa-sft/SKILL.md#L114) | | F5 | P2 | The host check apt-installs missing tools when run as root, which agents in containers saw without warning (LV2, cursor rows) | Stated in step 1 | SKILL.md [109–110](https://github.com/NVIDIA/cosmos-framework/blob/1ad7352ea/.agents/skills/cosmos3-qwen3-vl-robovqa-sft/SKILL.md#L109-L110) | | F6 | P2 | No troubleshooting entry for the new host-check message | "Things not obvious": what the banner labels mean and what to check | SKILL.md [222–225](https://github.com/NVIDIA/cosmos-framework/blob/1ad7352ea/.agents/skills/cosmos3-qwen3-vl-robovqa-sft/SKILL.md#L222-L225) | Left out on purpose: an evaluation-only `setup` path (three reports asked; a larger change, separate follow-up), and the review items unrelated to the eval (MD060 formatting, LFS clips, `docs/posttrain` revert, license text, reviewer comments on #343). ### Validation **Baseline (#343 @ affbdac)** — [dashboard group ci-pr343-affbdac](http://internal-cosmos-skills-eval.nvidia.com/skills/cosmos3-qwen3-vl-robovqa-sft/groups/ci-pr343-affbdac/): **8/8 PASS, 0/8 first-attempt** (every cell passed after one self-correction, F1). **This branch (`1ad7352`)** — manual CI-equivalent run on the eval runners (the eval workflow refuses fork PRs; same orchestrator, image and runners as CI), 2026-10-30 17:36–18:14 UTC: [results comment](#346 (comment)); dashboard groups [ci-pr346-1ad7352](http://internal-cosmos-skills-eval.nvidia.com/skills/cosmos3-qwen3-vl-robovqa-sft/groups/ci-pr346-1ad7352/) + [ci-pr346-1ad7352-run2](http://internal-cosmos-skills-eval.nvidia.com/skills/cosmos3-qwen3-vl-robovqa-sft/groups/ci-pr346-1ad7352-run2/) (the two ⛔ entries dated 17:28 UTC in the first group are a double-launch artifact, not a skill failure). **8/8 PASS, 8/8 first-attempt** (0 FAIL, 0 no report, 0 errors; static assessment 95/100). Mean overall score 87.8 → 97.4, correctness 80 → 100 on every cell, critical self-corrections 8 → 0, median cell time 30 → 15 minutes. **A** `group:ci-pr343-affbdac` (#343 head `affbdac`): 8/8 PASS, 0 first-attempt · **B** this PR (`1ad7352`): 8/8 PASS, 8 first-attempt · [dashboard compare, A vs this branch](http://internal-cosmos-skills-eval.nvidia.com/skills/cosmos3-qwen3-vl-robovqa-sft/compare?a=group%3Aci-pr343-affbdac&b=branch%3Askills%2Fqwen3-vl-robovqa-sft-eval-fixes) (branch scope = newest result per cell across both groups, all 8 cells; the run2 group alone lacks codex) | Case | Metric | claude | codex | cursor | opencode | |---|---|---|---|---|---| | `smoke-rows-and-preflight` | Completion | ✅→✅ | ✅→✅ | ✅→✅ | ✅→✅ | | | First time success | ❌→✅ | ❌→✅ | ❌→✅ | ❌→✅ | | `smoke-zero-shot-score` | Completion | ✅→✅ | ✅→✅ | ✅→✅ | ✅→✅ | | | First time success | ❌→✅ | ❌→✅ | ❌→✅ | ❌→✅ | 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Christina Wang <christwang@nvidia.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

Under set -o pipefail a trainer crash made the tee | grep | sed pipeline fail and the script exited there, so the "training did not finish" message after it never printed (review comment by Liang Feng on NVIDIA#343). The pipeline's status is now captured, and when the log lacks the end marker the step prints the status, the last 30 lines of the log and the log's path, then exits 1. Signed-off-by: sbabamohamad <sbabamohamad@nvidia.com>

christinayyw · 2026-10-30T22:46:45Z

Skill eval — cc8a437 · manual run (CI-equivalent)

8 / 8 cells done · 8 passed · 0 failed · 0 no report · 0 errors · 0 passed after self-correction · started 2026-10-30 21:45 UTC · updated 2026-10-30 22:42 UTC

`cosmos3-qwen3-vl-robovqa-sft` · dashboard overview

Case	claude	codex	cursor	opencode
`smoke-rows-and-preflight`	✅ PASS · report	✅ PASS · report	✅ PASS · report	✅ PASS · report
`smoke-zero-shot-score`	✅ PASS · report	✅ PASS · report	✅ PASS · report	✅ PASS · report

Legend: ✅ PASS · ❌ FAIL · ⚠️ PARTIAL · ❔ NO-REPORT (the agent finished without writing a report) · ⛔ ERROR (model gateway / agent / node error, see the report) · ↻ = passed after self-correction (the first attempt of the documented workflow failed; the agent fixed it and re-ran)

Before → after: affbdac (group ci-pr343-affbdac) 8/8 PASS, 0/8 on the first attempt → cc8a437 8/8 PASS, 8/8 on the first attempt · dashboard compare

Run details: this PR comes from a fork, which the self-hosted eval workflow does not run, so the same matrix was run by hand on the eval runners with the CI orchestrator (main) and image (cosmos-skill-eval:v20261102, evaluator 0.3.4), one backend per host, 1× H100 80GB (x86_64, driver 615.71.10). Static tier: 95/100.

✅ done — 8/8 passed (0 failed, 0 no report, 0 errors; 0 after self-correction)

christinayyw

Skill eval on cc8a437: 8/8 cells PASS, 8/8 on the first attempt across claude, codex, cursor and opencode (table and report links in #343 (comment); dashboard group ci-pr343-cc8a437). Approving.

sbabamohamad added 10 commits October 29, 2026 10:55

lfengad reviewed Oct 30, 2026

View reviewed changes

christinayyw and others added 2 commits October 30, 2026 10:25

christinayyw mentioned this pull request Oct 30, 2026

cosmos3-qwen3-vl-robovqa-sft: eval-derived fixes (stacked on #343) #346

Merged

yaoxu-crypto added the Skill New skill or updating existing skill label Oct 30, 2026

yaoxu-crypto approved these changes Oct 30, 2026

View reviewed changes

christinayyw approved these changes Oct 30, 2026

View reviewed changes

yaoxu-crypto enabled auto-merge (squash) October 30, 2026 22:48

yaoxu-crypto disabled auto-merge October 30, 2026 22:48

Merge branch 'main' into sft-qwen3-vl-robovqa

0316a21

yaoxu-crypto merged commit 50e8834 into NVIDIA:main Oct 31, 2026
13 checks passed

Navigation Menu

Repository navigation

skills: add cosmos3-qwen3-vl-robovqa-sft (Qwen3-VL-8B SFT on RoboVQA, one GPU, scored before and after) - #343

skills: add cosmos3-qwen3-vl-robovqa-sft (Qwen3-VL-8B SFT on RoboVQA, one GPU, scored before and after)#343
yaoxu-crypto merged 15 commits into
NVIDIA:mainfrom
sbabamohamad:sft-qwen3-vl-robovqa

sbabamohamad commented Oct 29, 2026

Uh oh!

christinayyw commented Oct 30, 2026

Uh oh!

lfengad left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

christinayyw commented Oct 30, 2026

Uh oh!

christinayyw left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

sbabamohamad commented Oct 29, 2026

What this adds

Results (one GB300, two full runs at seed 0, framework 9a70e64)

Testing

Eval cases

Uh oh!

christinayyw commented Oct 30, 2026

Skill eval — affbdac · manual run (CI-equivalent)

cosmos3-qwen3-vl-robovqa-sft · dashboard overview

Uh oh!

lfengad left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

christinayyw commented Oct 30, 2026

Skill eval — cc8a437 · manual run (CI-equivalent)

cosmos3-qwen3-vl-robovqa-sft · dashboard overview

Uh oh!

christinayyw left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Results (one GB300, two full runs at seed 0, framework `9a70e64`)

`cosmos3-qwen3-vl-robovqa-sft` · dashboard overview

`cosmos3-qwen3-vl-robovqa-sft` · dashboard overview