Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions docs/design/declarative_expert_program_plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -1270,6 +1270,33 @@ validator and parallel physical integration remain pending. The PourWater task
migration is outside the current scope because it would require modifying
Action Bank code.

The current follow-up also makes task registration the sole standard-runtime
extension owner. `SkillPolicyPreset` schema version 2 requires exact typed
action-option templates for every reachable semantic call; lowering may fill
only explicitly compiler-owned dynamic target fields. Endpoint adapters,
ordered Gym transports, and a parallel-safety factory are declared on
`SimulationExpertProgramRegistration`, enter its provider-free fingerprint,
and are cross-checked again against live endpoint resolution. The standard
factory consumes the same registration objects, freezes the assembled command
encoder, takes runner timing from the selected preset, and creates a fresh live
safety validator for every runtime assembly. No helper argument can replace
those registered components after preflight. Stateful extension declarations
must be frozen dataclasses with recursively immutable configuration, preventing
nested mutable values from becoming a post-registration runtime side channel.

This registration slice deliberately covers command transport, not arbitrary
closed-loop backend injection. In C1, every custom endpoint adapter must declare
empty tracking and effect-evidence route sets and therefore supports only
timed/open-loop completion. The built-in `ControlPartEndpoint` retains its exact
built-in routes. A non-joint feedback provider, desired-state projector, metric
evaluator, or effect-evidence backend needs a separate registration-owned
live-provider factory contract before it can be advertised as standard
mobile/whole-body closed-loop support. Such providers must become fingerprinted
capabilities; they must not return as task-side runtime callbacks. Transport
`hold()` remains a trusted safe primitive owned and tested by each transport,
while the parallel safety validator authorizes active merged command frames
before dispatch.

Deliverables:

- `Parallel` and explicit `Barrier` nodes in a new schema version;
Expand Down
39 changes: 31 additions & 8 deletions docs/source/overview/sim/atomic_actions/expert_programs.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,13 +127,18 @@ task then delegates runtime assembly to the shared factory; it does not
construct approach, grasp, pull, or placement trajectories:

```python
MY_EXPERT_PROGRAM_REGISTRATION = SimulationExpertProgramRegistration(
scene_binding=create_my_scene_binding(),
robot_profile_binding=create_my_robot_profile_binding(),
)


class MyTaskEnv(ExpertProgramEnvironmentMixin, EmbodiedEnv):
def __init__(self, cfg, **kwargs):
super().__init__(cfg, **kwargs)
self._expert_program_adapter = create_simulation_expert_program_adapter(
self,
scene_binding=create_my_scene_binding(),
robot_profile_binding=create_my_robot_profile_binding(),
registration=MY_EXPERT_PROGRAM_REGISTRATION,
)

@property
Expand All @@ -149,8 +154,20 @@ monitor selection. `SimulationRobotSkillProfileBinding` accepts generic
`ResourceEndpoint` values; `ControlPartResourceBinding` is its stricter
joint-backed convenience. Endpoint adapters and runtime transports are the
extension boundary for mobile-base, whole-body, or non-joint controllers and
are accepted by the standard simulation helper. Task programs keep the same
semantic calls and do not gain controller-shaped fields.
are owned by `SimulationExpertProgramRegistration`, not passed as live helper
overrides. Their exact static target, payload, route, and transport declarations
enter the catalog fingerprint, while transport tuple order defines deterministic
Gym-action composition order. Task programs keep the same semantic calls and do
not gain controller-shaped fields.

The current standard registration installs built-in joint tracking and effect
evidence providers only for `ControlPartEndpoint`. Every custom endpoint adapter
must declare empty tracking/evidence route sets and therefore uses
timed/open-loop completion. A non-joint closed-loop projector, feedback source,
or effect-evidence backend still requires the planned registration-owned
provider-factory extension; it must not be injected from a task after preflight.
Whole-body controllers expressed through existing joint control parts continue
to use the built-in joint route.

Relation and rendezvous semantics are also explicit integration capabilities.
`Place(on=...)` and `Place(inside=...)` require an exact typed/versioned
Expand Down Expand Up @@ -185,9 +202,12 @@ of evidence:
- the live object-to-endpoint pose relation from the shared scene snapshot.

The command-state update is transactional: encoder, buffer, cancellation, or
safe-stop failures invalidate it. An integration with contact, constraint,
force, or wrench sensing can install typed evidence callbacks without changing
the semantic call or program.
safe-stop failures invalidate it. The current C1 standard path does not accept
task-side evidence callbacks: custom endpoint adapters must expose empty
tracking and effect-evidence route sets. Contact, constraint, force, wrench, or
other custom closed-loop sensing requires a future registration-owned provider
factory whose declaration enters the integration fingerprint; this will not
change the semantic call or program.

Program/demo-segment metadata records runtime call results, named trajectory
segments, effect decisions, recovery events, scene and collision revisions,
Expand All @@ -199,7 +219,10 @@ Schema-version-2 parallel blocks additionally require an authoritative
`ParallelCommandSafetyValidator`. Resource-claim disjointness is necessary but
is not treated as proof of physical safety. If no validator is installed, the
parallel block refuses to start; the standard simulation adapter intentionally
does not invent one from resource names. Every parallel frame must occupy
does not invent one from resource names. Its task registration must instead
declare a safety factory covering the exact registered transport set; each
runtime assembly receives a fresh validator instance from that factory. Every
parallel frame must occupy
exactly one `BaseEnv.step_dt`; shorter lanes repeat their last safe target as
hold padding, while fractional frames are rejected rather than resampled.
Version 2 also uses strict symbolic key-level conflict detection at the barrier:
Expand Down
58 changes: 54 additions & 4 deletions docs/source/overview/sim/atomic_actions/robot_skill_profiles.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,10 @@ from embodichain.lab.sim.atomic_actions import (
FORWARD_KINEMATICS_CAPABILITY,
GRASP_CAPABILITY,
ControlPartCommandProfile,
HandOverOptions,
MotionPolicy,
PickUpOptions,
PlaceOptions,
)
from embodichain.lab.sim.skills import (
COMPOSITE_EFFECT_MONITOR_ID,
Expand Down Expand Up @@ -140,6 +143,11 @@ profile = RobotSkillProfile(
presets={
"default": SkillPolicyPreset(
preset_id="default",
action_option_templates={
"pick": PickUpOptions(),
"place": PlaceOptions(),
"hand_over": HandOverOptions(),
},
motion_policy=MotionPolicy(strategy="ik_interp"),
effect_monitors={
semantic_id: EffectMonitorRef(
Expand Down Expand Up @@ -195,6 +203,28 @@ A linked call receives an effective immutable preset snapshot with
Other presets, and scenes without dynamic collision entities, retain their
configured collision mode.

## Configure semantic action behavior with the preset

`SkillPolicyPreset.action_option_templates` is the required, typed action-
behavior table for semantic calls that can select the preset. Each key is the
exact semantic call ID (`pick`, `place`, `hand_over`, or
`operate_articulation`), and each value must be the target action's exact frozen
`ActionOptions` dataclass. Static linking rejects a missing entry, an unknown
call ID, or an options value of the wrong exact type before simulation starts.

The preset owns independent snapshots of each template. Pick and HandOver
grounding only replace their compiler-owned dynamic target fields; distances,
directions, waypoint counts, and other reusable behavior remain configuration.
A registered semantic lowerer may build a goal but cannot return replacement
options. This keeps task extensions from silently moving action parameters back
into Python code.

Pick's `downstream_object_target_poses` and HandOver's
`middle_object_pose`/`final_object_pose` are reserved for the semantic compiler
and must remain empty in a template. Planner choice, sample count, tracking,
recovery, runner timing, and effect monitors stay in their dedicated preset
fields rather than `ActionOptions`.

## Select semantic effect monitors with the preset

A {class}`SkillPolicyPreset` owns one coherent runtime choice: planning and
Expand Down Expand Up @@ -357,13 +387,33 @@ profile = SimulationRobotSkillProfileBinding(

adapter = create_simulation_expert_program_adapter(
env,
scene_binding=scene_binding,
robot_profile_binding=profile,
endpoint_adapters={MobileVelocityEndpoint: MobileVelocityEndpointAdapter()},
runtime_transports=(MobileVelocityGymEncoder(),),
registration=SimulationExpertProgramRegistration(
scene_binding=scene_binding,
robot_profile_binding=profile,
endpoint_adapters=(MobileVelocityEndpointAdapter(),),
runtime_transports=(MobileVelocityGymEncoder(),),
),
)
```

The adapter and encoder publish exact class-level declarations before a live
robot is created. The adapter declares its endpoint type, runtime target types,
transport IDs, and versioned tracking/evidence routes. The encoder declares its
transport ID plus exact target and payload types; each target and payload type
declares the same `TRANSPORT_ID`. Registration rejects missing, unused,
duplicate, or conflicting declarations, and runtime profile binding verifies
that `adapter.resolve()` returns only those declared routes. A stateful adapter,
transport, grounding provider, or safety factory must be a frozen dataclass whose
configuration is recursively immutable; mutable leaves such as lists, mappings,
sets, byte arrays, and tensors are rejected before registration.

The standard factory currently accepts its built-in tracking feedback,
projector, evaluator, and effect-evidence routes only for
`ControlPartEndpoint`. In C1, every custom endpoint adapter must declare empty
route sets and therefore supports timed/open-loop execution only. Custom
closed-loop mobile or whole-body tracking/evidence needs a registration-owned
provider factory in C2; task code must not supply a live provider side channel.

`RobotResourceBinding` snapshots arbitrary typed `ResourceEndpoint` values.
`ControlPartResourceBinding` remains the stricter joint-backed convenience and
continues to validate native control parts, joint IDs, and command-preset
Expand Down
28 changes: 27 additions & 1 deletion embodichain/lab/gym/envs/expert_program/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,19 @@
SimulationRobotSkillProfileBinding,
SimulationSceneBinding,
)
from .catalog import (
ExpertProgramIntegrationCatalog,
IntegrationFingerprintMismatch,
SimulationExpertProgramRegistration,
)
from .extensions import (
EndpointAdapterDeclaration,
ParallelCommandSafetyValidatorFactory,
ParallelSafetyDeclaration,
RuntimeTransportDeclaration,
StandardExtensionDeclarations,
VersionedKey,
)
from .simulation_environment import (
ControlCommandStateEvidenceTracker,
MotionGeneratorFactory,
Expand All @@ -139,7 +152,10 @@
SimulationPlanningObservationProvider,
create_simulation_expert_program_adapter,
)
from .simulation_policies import SimulationSegmentPolicyPort
from .simulation_policies import (
SimulationSegmentPolicyPort,
default_simulation_settle_presets,
)

__all__ = [
"AcceptedRuntimeCommandObserver",
Expand Down Expand Up @@ -174,6 +190,7 @@
"EXPERT_PROGRAM_SCHEMA_VERSION_V2",
"EnvironmentStepClock",
"EnvironmentStepTimingError",
"EndpointAdapterDeclaration",
"ExpertProgramCfg",
"ExpertProgramCompileError",
"ExpertProgramCompiler",
Expand All @@ -183,13 +200,15 @@
"ExpertProgramEnvironmentFactory",
"ExpertProgramEnvironmentMixin",
"ExpertProgramIntegrationCfg",
"ExpertProgramIntegrationCatalog",
"ExpertProgramRuntimeAssembly",
"ExpertProgramSceneResolver",
"ExpertProgramValidationContext",
"ExpertProgramValidationError",
"HandOverCfg",
"GymPlanningObservationProvider",
"InvokeCfg",
"IntegrationFingerprintMismatch",
"MAX_DECLARATIVE_DEPTH",
"MAX_DECLARATIVE_NODES",
"MAX_EXPANDED_CALLS",
Expand All @@ -202,6 +221,8 @@
"ObjectNearTargetValidatorCfg",
"OperateArticulationCfg",
"ParallelCfg",
"ParallelCommandSafetyValidatorFactory",
"ParallelSafetyDeclaration",
"PickCfg",
"PlaceCfg",
"PlanningObservationPort",
Expand All @@ -212,6 +233,7 @@
"RepeatCfg",
"RobotResourceBinding",
"RuntimeCommandFrameEncoder",
"RuntimeTransportDeclaration",
"RuntimeTransportActionEncoder",
"SceneReferenceRole",
"SceneRegistryProgramResolver",
Expand All @@ -228,20 +250,24 @@
"SimulationArticulationLinkBinding",
"SimulationExpertProgramEnvironment",
"SimulationExpertProgramFactory",
"SimulationExpertProgramRegistration",
"SimulationPlanningObservationProvider",
"SimulationRigidObjectBinding",
"SimulationResourceEndpointBinding",
"SimulationRobotResourceBinding",
"SimulationRobotSkillProfileBinding",
"SimulationSceneBinding",
"SimulationSegmentPolicyPort",
"StandardExtensionDeclarations",
"SUPPORTED_EXPERT_PROGRAM_SCHEMA_VERSIONS",
"TargetCfg",
"TargetRefCfg",
"UnsupportedRuntimeTransportError",
"ValidatorCfg",
"VersionedKey",
"WaitStablePostCfg",
"create_simulation_expert_program_adapter",
"default_simulation_settle_presets",
"decode_expert_program",
"load_expert_program",
"loads_expert_program_json",
Expand Down
Loading