Repository navigation
Action channel layout + normalizers for egocentric (57D) and robot embodiments #184
Description
Thanks for the release. I'm running action-conditioned inference with Cosmos3-Nano via vLLM-Omni. The model card gives per-embodiment dims and says extra_params["action"] takes a normalized (T, D) array, but only agibotworld (FD) and av (ID) examples ship, and the exact channel layout + normalizers aren't documented. Could you share, ideally as a per-embodiment table:
- Egocentric (57D): exact ordering — block order (ego 9D / left vs right hand / wrist 9D + grasp 15D), translation-vs-rotation order in each 9D pose, the 6D-rotation layout, the 5-fingertip order and which of the 21 keypoints they are, coordinate convention, and the
domain_namestring. - Robots (Franka 10D, dual Franka 20D, AgiBot 29D, UR/Google/WidowX 10D, UMI 9D): exact channel order (EE translation/rotation/gripper) — pose-delta or joint-angle based?
- Normalizers: where the per-dimension stats live and how to apply them.
A pointer to the canonical file in cosmos-framework would be perfect. Thanks!
Activity
Huaian666 commentedon Jun 4, 2026
really impressive work from nvidia. the world model approach makes total sense for physical ai. looking forward to seeing where this goes.
mli0603 commentedon Jun 4, 2026
Thanks @TQTQliu for the question. We are working on releasing more examples similar to DROID that
And we are working on both FD/ID. Will provide an update.
Rohithmatham12 commentedon Jun 13, 2026
I opened #212 as a small documentation follow-up related to this request. It does not attempt to document unreleased embodiment-specific layouts or normalizer files, but it clarifies that the current canonical references are the checked-in AV, DROID, and UMI action assets/notebooks, and points users to the DROID LeRobot sample metadata for the available state/action relationship.
That should make the current boundary clearer while NVIDIA prepares the additional examples and normalization files mentioned above.