feat(tasks): add declarative expert program vertical slices - #499
feat(tasks): add declarative expert program vertical slices#499yuecideng wants to merge 1 commit into
Conversation
Greptile SummaryThe PR migrates repeated cube pick/place and Open Drawer demonstrations from task-owned motion generation to declarative Expert Programs backed by typed scene and robot-profile bindings.
Confidence Score: 5/5The PR appears safe to merge, with no concrete blocking or independently actionable non-blocking issue identified. The declarative program identifiers align with their typed task bindings, configuration paths resolve through the supported loading paths, and the investigated initialization, binding, and test-security concerns did not establish observable failures.
|
| Filename | Overview |
|---|---|
| embodichain_tasks/embodichain_tasks/multi_segments/cube_pick_place.py | Replaces task-owned pick/place trajectory and validation logic with typed cube scene/profile declarations and the shared Expert Program adapter. |
| embodichain_tasks/embodichain_tasks/tableware/open_drawer.py | Replaces the fixed drawer-opening trajectory with typed articulation geometry, robot resources, command presets, and shared semantic runtime integration. |
| embodichain_tasks/configs/expert_program/multi_segments/repeated_cube_pick_place.yaml | Declares three repeated cube pick/place segments with cyclic targets, settling, and target-proximity validation. |
| embodichain_tasks/configs/expert_program/tableware/open_drawer.json | Declares one articulation-operation segment targeting the drawer’s named open state. |
| embodichain_tasks/configs/gym/multi_segments/cube_pick_place.json | Selects the packaged cube Expert Program and adds a bounded reset-settling event. |
| tests/gym/envs/expert_program/test_task_vertical_slices.py | Adds focused coverage for the two declarative task integrations and their shared runtime path. |
| tests/test_expert_program_package_data.py | Verifies that the added Expert Programs are included and decode correctly from staged package data. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
A[Gym task config] --> B[Packaged Expert Program]
C[Task scene binding] --> D[Simulation Expert Program adapter]
E[Task robot profile binding] --> D
B --> D
D --> F[Semantic compiler and runtime]
F --> G[Buffered commands through env.step]
G --> H[Effect monitoring]
H --> I[Settling and validation metadata]
Reviews (1): Last reviewed commit: "feat(tasks): add declarative expert prog..." | Re-trigger Greptile
There was a problem hiding this comment.
Pull request overview
This PR migrates two existing simulation tasks (multi-segment cube pick/place and open drawer) from task-owned waypoint/motion planning code to declarative Expert Programs (JSON/YAML) backed by typed scene + robot-profile bindings, validating that tasks can be expanded “configuration-only” when the underlying semantic capability already exists.
Changes:
- Convert
MultiSegmentsCubePickPlaceEnvandOpenDrawerEnvto useExpertProgramEnvironmentMixin+create_simulation_expert_program_adapter, leaving tasks responsible only for scene/profile declarations. - Add packaged Expert Program resources (
repeated_cube_pick_place.yaml,open_drawer.json) and update Gym configs to reference them viaexpert_program_path. - Add focused vertical-slice tests and update documentation/context to cover Expert Programs, effect evidence/monitoring, and articulation operation semantics.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/test_expert_program_package_data.py | Verifies setuptools stages Expert Program JSON/YAML and that staged resources decode via installed config paths. |
| tests/gym/envs/tasks/test_open_drawer.py | Validates OpenDrawer task registration, config wiring to packaged program, adapter construction, and (optional) real-sim execution evidence. |
| tests/gym/envs/tasks/test_multi_segments_cube_pick_place.py | Validates cube task registration, config wiring to packaged program, adapter construction, and profile/settle configuration. |
| tests/gym/envs/expert_program/test_task_vertical_slices.py | End-to-end non-physical vertical slices: strict decode/compile, lazy lifecycles, settle/validator metadata, and task “no motion override” guarantees. |
| embodichain_tasks/embodichain_tasks/tableware/open_drawer.py | Replaces task-local motion generation with typed scene/profile declarations for operate_articulation. |
| embodichain_tasks/embodichain_tasks/multi_segments/cube_pick_place.py | Replaces task-local lazy planning/settling with typed scene/profile declarations plus packaged repeated-cube Expert Program. |
| embodichain_tasks/configs/gym/open_drawer/cobot_magic_3cam.json | Adds expert_program_path to select the packaged drawer program. |
| embodichain_tasks/configs/gym/multi_segments/cube_pick_place.json | Adds expert_program_path, moves settling to reset events, and trims extensions to semantic-only knobs. |
| embodichain_tasks/configs/expert_program/tableware/open_drawer.json | Adds the declarative Open Drawer Expert Program (segment invoking operate_articulation). |
| embodichain_tasks/configs/expert_program/multi_segments/repeated_cube_pick_place.yaml | Adds the declarative repeated cube pick/place Expert Program (repeat of segment with post-policy + validator). |
| docs/source/tutorial/atomic_actions.rst | Documents scene-dependency monitoring cutoffs via scene_dependency_monitor_until. |
| docs/source/overview/sim/index.rst | Adds navigation guidance for when to use Expert Programs. |
| docs/source/overview/sim/atomic_actions/robot_skill_profiles.md | Extends docs for effect monitors, endpoint metadata, and Expert Program integration patterns. |
| docs/source/overview/sim/atomic_actions/index.md | Adds Expert Programs doc to the atomic-actions overview and clarifies effect evidence vs decision responsibilities. |
| docs/source/overview/sim/atomic_actions/builtin_actions.md | Documents operate_articulation and its dependency-monitor cutoff semantics. |
| docs/design/declarative_expert_program_plan.md | Updates project design plan status/progress notes for the stacked implementation. |
| agent_context/topics/atomic-actions/atomic-actions.md | Updates agent-facing context with semantic runtime + Expert Program details. |
| agent_context/MAP.yaml | Expands topic keywords and file pointers for Expert Program / semantic runtime content. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| environment = os.environ.copy() | ||
| environment["PYTHONPATH"] = str(staged_config_package.build_lib) | ||
| completed = subprocess.run( |
Description
Stack
feat/mllm-expert-program-frontendMigrate repeated Cube Pick/Place and Open Drawer to declarative Expert Programs with typed scene/profile integration. Task classes no longer own waypoint math, invocation assembly, or demo-generation overrides.
The two vertical slices prove configuration-only task expansion when the shared semantic capability already exists, while keeping task validation and physical settling explicit.
Refs #471
Refs #474
Type of change
Screenshots
Not applicable.
Validation
tests/gym/envs/expert_program/test_task_vertical_slices.py, cube/open-drawer task tests, and package-data testsChecklist