Repository navigation
Unable to reproduce nano droid policy post-training #138
Description
I followed your post-training configs and used your released dataset, but I can get 0.1783 SR.
I noticed that there is a f_max=0.4 that restricts the lr to 8e-5 instead of 2e-4, and the lr scheduler cycle is 100k instead of 10k.
Are these parameters intended?
Did you test your released toml configs?
(model.rectified_flow_training_config and dataloader_train.dataloader are not registered in SFTExperimentConfig, so it contradicts the extra="forbid" and raises errors.)
Reacted by shijun
Metadata
Metadata
Assignees
Labels
No labels
Activity
fix(action): make nano DROID policy SFT TOML validate against the schema
fwd4 commentedon Jul 28, 2026
Both are intended, and those are used our Nano model training. @ychao-nvidia
This is a real issue, resolved in PR139
Randdl commentedon Jul 28, 2026
Thanks a lot! Can you also verify the configs with your internal post-training configs? Are there any silently missing values or values overwritten by defaults? The two unregistered values suggest there might be other misalignments.
Also, I think the robolab eval with your checkpoint silently falls back to plain text prompt; however, the post-training config you provided uses JSON format. Could you confirm if this is the case or if I used it wrong? This is the only reason I can find that might explain my bad reproduction result right now.
fix(action): validate nano DROID policy SFT TOML against the schema (#…
fwd4 commentedon Jul 28, 2026
Thank you for bring this up. Everything else in the config is a match. One thing you might want to handle carefully is the range filter, make sure you are following the steps in the doc.
Yes, we need post-trained Nano and Edge policy model both having json prompt. But normally it would be sr drop of less than 10pp even if you have not configured json/dense format properly. @ychao-nvidia can you confirm on this on the server script?
Randdl commentedon Jul 28, 2026
I used the range filter, so I don't think this is my root cause.
What I found is that your Nano checkpoint uses plain text, but edge uses json format. During evaluation on Robolab (not training), when I force nano to use json format, the sr rate actually drops from 0.3975 to 0.2050.
fwd4 commentedon Jul 28, 2026
@Randdl noted. We should take a look on the released checkpoint. Can you kindly evaluate your trained ckpt with forced json prompt?
ychao-nvidia commentedon Jul 28, 2026
@Randdl
This is intended for our released checkpoints. The released Nano-Policy-DROID was trained from an earlier experiment that used dense prompt. We switched all our training to json prompt later.
Randdl commentedon Jul 28, 2026
I trained my ckpt with json prompt. When I evaluate it with dense prompt, the SR drops. I think other than these, I am following the post-training configs faithfully.
Randdl commentedon Jul 28, 2026
Two more distinctions I found.
The nano droid checkpoint's reasoner weights deviate from the nano checkpoint, and is identical to the vanilla Qwen weights.
The other action heads (untrained) in nano droid checkpoints are identical to the nano checkpoint. However, if following the post-training config, all the action heads will be re-initialized. It suggests the nano droid checkpoint did not re-initialize the action heads.
Randdl commentedon Aug 1, 2026
keephead and vanilla qwen did not help. Do you have any updates?
Your released checkpoint apparently does not align with the config you released. Can you provide the real config you used for the checkpoint or provide a checkpoint that you reproduce using the config?
fwd4 commentedon Aug 3, 2026
Hi @Randdl
As @ychao-nvidia mentioned, the policy checkpoint released is not trained from the released Cosmos3-Nano model, but from an internal midtrain checkpoint. That would explain reasoner weights are different in these two models.
We haven't updated our Cosmos3-Nano-Droid-Policy ckpt yet so the config mismatch is expected. The config in this repo is expected to reproduce the SR with json prompts, training from Cosmos3-Nano. Let me know if you have trouble reproducing json prompt evaluation results on your trained ckpt.
Randdl commentedon Aug 3, 2026
Hi @fwd4
I'm using json prompt. I can only get SR=0.25 using the official config. The SR seems to oscillate a lot with iterations.
iters | SR
8000 | 0.24
8500 | 0.23
9000 | 0.26
9500 | 0.18
1000 | 0.18
fwd4 commentedon Aug 4, 2026
@Randdl This would be too much, normally Robolab has 1-2pp variation among runs. Since you have secured ~40% SR on the official ckpt, I think we can rule out evaluation problem. Can you provide:
Randdl commentedon Aug 4, 2026
@fwd4 Do you mind if I talk through emails with you? It will be easier to send training details via email.
I used 256 GB300, global batch size 8192. lr is going from 8e-5 to 7.2e-5. Loss is pretty noisy. Between 8000-10000 iterations, it is between 1.00-1.67 with std 0.153. action loss mean 0.0023, sd 0.0013. vision loss mean 0.13, sd 0.015.
I did get a better SR=0.28 by using vanilla Qwen as the reasoner weights, but I don't think that's the root cause.
fwd4 commentedon Aug 4, 2026
Sure, let me ping you @Randdl
fwd4 commentedon Aug 4, 2026
I checked the loss curve you provided, the loss is noisy but averagely tracking our loss curve, and the LR curve looks exactly the same too. One more issue, are you evaluating with
net_emainstead ofnet? If not, please try use the ema weights. That might also explain the big SR fluctuations among iterations.Randdl commentedon Aug 5, 2026
I got SR=0.37 after using EMA weights. Thanks! It seems that my previous ema on setting wasn't used.
AIprogrammer commentedon Aug 6, 2026
Hi, could you also share the loss curve on your side for reference?
Regarding EMA, just to confirm: do you mean the EMA weights accumulated during training (i.e., evaluating net_ema instead of the raw net checkpoint)? If so, is it expected that this alone can lead to such a large gap in SR? The improvement from the non-EMA weights to EMA weights in this case seems quite substantial, so we would like to better understand whether this is typical or if there may be another setting difference involved as well.
Sync/main into kunlun dev 20260804 (#16)