Skip to content

feat(tasks): add declarative expert program vertical slices - #499

Closed
yuecideng wants to merge 1 commit into
feat/mllm-expert-program-frontendfrom
feat/declarative-task-vertical-slices
Closed

feat(tasks): add declarative expert program vertical slices#499
yuecideng wants to merge 1 commit into
feat/mllm-expert-program-frontendfrom
feat/declarative-task-vertical-slices

Conversation

@yuecideng

@yuecideng yuecideng commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Description

Stack

Migrate repeated Cube Pick/Place and Open Drawer to declarative Expert Programs with typed scene/profile integration. Task classes no longer own waypoint math, invocation assembly, or demo-generation overrides.

The two vertical slices prove configuration-only task expansion when the shared semantic capability already exists, while keeping task validation and physical settling explicit.

Refs #471
Refs #474

Type of change

  • New feature (non-breaking change which adds functionality)

Screenshots

Not applicable.

Validation

  • Focused coverage: tests/gym/envs/expert_program/test_task_vertical_slices.py, cube/open-drawer task tests, and package-data tests
  • Final affected-suite regression on the stack tip: 1215 passed, 2 skipped, 8 deselected
  • Changed Python files pass Black 26.3.1; the Sphinx build and rollout-report drift check pass at the stack tip

Checklist

  • Changed Python files pass Black 26.3.1.
  • Corresponding public/design documentation is included in this stack.
  • Tests cover the affected behavior.
  • No dependency update is required.

@yuecideng yuecideng added task A task written in openai gym format for imitation learning or reinforcement learning gym robot learning env and its related features enhancement New feature or request labels Aug 11, 2026
@yuecideng
yuecideng marked this pull request as ready for review August 11, 2026 16:46
Copilot AI lite review requested due to automatic review settings August 11, 2026 16:46
@greptile-apps

greptile-apps Bot commented Aug 11, 2026

Copy link
Copy Markdown

Greptile Summary

The PR migrates repeated cube pick/place and Open Drawer demonstrations from task-owned motion generation to declarative Expert Programs backed by typed scene and robot-profile bindings.

  • Adds packaged JSON/YAML Expert Programs and selects them from the corresponding Gym configurations.
  • Replaces task-specific trajectory assembly with shared simulation Expert Program adapters.
  • Adds reset settling for the cube task and declarative post-settling and validation behavior.
  • Expands focused task, vertical-slice, and package-data coverage.
  • Updates architectural and user documentation for the semantic runtime, effect monitoring, articulation operations, and Expert Programs.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete blocking or independently actionable non-blocking issue identified.

The declarative program identifiers align with their typed task bindings, configuration paths resolve through the supported loading paths, and the investigated initialization, binding, and test-security concerns did not establish observable failures.

Important Files Changed

Filename Overview
embodichain_tasks/embodichain_tasks/multi_segments/cube_pick_place.py Replaces task-owned pick/place trajectory and validation logic with typed cube scene/profile declarations and the shared Expert Program adapter.
embodichain_tasks/embodichain_tasks/tableware/open_drawer.py Replaces the fixed drawer-opening trajectory with typed articulation geometry, robot resources, command presets, and shared semantic runtime integration.
embodichain_tasks/configs/expert_program/multi_segments/repeated_cube_pick_place.yaml Declares three repeated cube pick/place segments with cyclic targets, settling, and target-proximity validation.
embodichain_tasks/configs/expert_program/tableware/open_drawer.json Declares one articulation-operation segment targeting the drawer’s named open state.
embodichain_tasks/configs/gym/multi_segments/cube_pick_place.json Selects the packaged cube Expert Program and adds a bounded reset-settling event.
tests/gym/envs/expert_program/test_task_vertical_slices.py Adds focused coverage for the two declarative task integrations and their shared runtime path.
tests/test_expert_program_package_data.py Verifies that the added Expert Programs are included and decode correctly from staged package data.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[Gym task config] --> B[Packaged Expert Program]
    C[Task scene binding] --> D[Simulation Expert Program adapter]
    E[Task robot profile binding] --> D
    B --> D
    D --> F[Semantic compiler and runtime]
    F --> G[Buffered commands through env.step]
    G --> H[Effect monitoring]
    H --> I[Settling and validation metadata]
Loading

Reviews (1): Last reviewed commit: "feat(tasks): add declarative expert prog..." | Re-trigger Greptile

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates two existing simulation tasks (multi-segment cube pick/place and open drawer) from task-owned waypoint/motion planning code to declarative Expert Programs (JSON/YAML) backed by typed scene + robot-profile bindings, validating that tasks can be expanded “configuration-only” when the underlying semantic capability already exists.

Changes:

  • Convert MultiSegmentsCubePickPlaceEnv and OpenDrawerEnv to use ExpertProgramEnvironmentMixin + create_simulation_expert_program_adapter, leaving tasks responsible only for scene/profile declarations.
  • Add packaged Expert Program resources (repeated_cube_pick_place.yaml, open_drawer.json) and update Gym configs to reference them via expert_program_path.
  • Add focused vertical-slice tests and update documentation/context to cover Expert Programs, effect evidence/monitoring, and articulation operation semantics.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/test_expert_program_package_data.py Verifies setuptools stages Expert Program JSON/YAML and that staged resources decode via installed config paths.
tests/gym/envs/tasks/test_open_drawer.py Validates OpenDrawer task registration, config wiring to packaged program, adapter construction, and (optional) real-sim execution evidence.
tests/gym/envs/tasks/test_multi_segments_cube_pick_place.py Validates cube task registration, config wiring to packaged program, adapter construction, and profile/settle configuration.
tests/gym/envs/expert_program/test_task_vertical_slices.py End-to-end non-physical vertical slices: strict decode/compile, lazy lifecycles, settle/validator metadata, and task “no motion override” guarantees.
embodichain_tasks/embodichain_tasks/tableware/open_drawer.py Replaces task-local motion generation with typed scene/profile declarations for operate_articulation.
embodichain_tasks/embodichain_tasks/multi_segments/cube_pick_place.py Replaces task-local lazy planning/settling with typed scene/profile declarations plus packaged repeated-cube Expert Program.
embodichain_tasks/configs/gym/open_drawer/cobot_magic_3cam.json Adds expert_program_path to select the packaged drawer program.
embodichain_tasks/configs/gym/multi_segments/cube_pick_place.json Adds expert_program_path, moves settling to reset events, and trims extensions to semantic-only knobs.
embodichain_tasks/configs/expert_program/tableware/open_drawer.json Adds the declarative Open Drawer Expert Program (segment invoking operate_articulation).
embodichain_tasks/configs/expert_program/multi_segments/repeated_cube_pick_place.yaml Adds the declarative repeated cube pick/place Expert Program (repeat of segment with post-policy + validator).
docs/source/tutorial/atomic_actions.rst Documents scene-dependency monitoring cutoffs via scene_dependency_monitor_until.
docs/source/overview/sim/index.rst Adds navigation guidance for when to use Expert Programs.
docs/source/overview/sim/atomic_actions/robot_skill_profiles.md Extends docs for effect monitors, endpoint metadata, and Expert Program integration patterns.
docs/source/overview/sim/atomic_actions/index.md Adds Expert Programs doc to the atomic-actions overview and clarifies effect evidence vs decision responsibilities.
docs/source/overview/sim/atomic_actions/builtin_actions.md Documents operate_articulation and its dependency-monitor cutoff semantics.
docs/design/declarative_expert_program_plan.md Updates project design plan status/progress notes for the stacked implementation.
agent_context/topics/atomic-actions/atomic-actions.md Updates agent-facing context with semantic runtime + Expert Program details.
agent_context/MAP.yaml Expands topic keywords and file pointers for Expert Program / semantic runtime content.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +179 to +181
environment = os.environ.copy()
environment["PYTHONPATH"] = str(staged_config_package.build_lib)
completed = subprocess.run(
@yuecideng

Copy link
Copy Markdown
Contributor Author

Folded into #500 during stacked-PR consolidation. Its commits remain included in #500; the remote branch is retained for traceability.

@yuecideng yuecideng closed this Aug 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request gym robot learning env and its related features task A task written in openai gym format for imitation learning or reinforcement learning

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants