feat: libero autoresearch

[TomCC7]

Contribution path

• Stacked on feat(eval): integrate LIBERO-PRO code-policy benchmark #3439.
• Merge feat(eval): integrate LIBERO-PRO code-policy benchmark #3439 before this PR.

Problem

The initial LIBERO-PRO integration exposes calibrated observations and ordinary manipulation RPCs, but realtime code policies still have to assemble grounding, depth projection, grasp geometry, gripper control, placement, and verification correctly within a short simulator horizon. In repeated trials, policies often selected the right object yet failed to retain it, released it against a receptacle wall, or blocked the policy session with an overlong kernel call.

We need general manipulation capability developed through LIBERO, not benchmark-specific coordinates or task logic.

Solution

Add a general explicit-observation manipulation path on top of #3439:

• add point-grounded segment_best perception and calibrated 2D-to-3D projection typing
• add GraspExecutionModule RPCs for ranked grasp execution and point-cloud pick-and-place
• accept the policy-selected RGBDObservation explicitly; the module does not subscribe to a hidden camera stream or access privileged evaluation state
• align the Panda closing axis from observed object geometry, verify retained gripper aperture after retract, and retry empty grasps
• place held objects at the center of observed receptacle bounds, just below the rim, then settle and retreat
• cap individual CodePolicy kernel calls below the transport timeout while retaining the overall exploration budget
• return policy output in trial responses and strengthen the LIBERO prompt around direct image inspection and ordinary manipulation APIs
• add frozen four-family development and held-out panels plus an autoresearch runner for repeatable native-score experiments

The retained implementation contains no LIBERO object names, task IDs, or hard-coded scene coordinates.

Autoresearch improved the frozen development panel from 0/4 to 1/4 native success with zero infrastructure/policy failures. The object success reproduced unchanged in a second full-panel run. Both successful trials ended through LIBERO's native BDDL predicate with score 1.0 and terminal reason success.

How to Test

Run the focused suite:

uv run pytest \
  dimos/agents/test_code_policy_core.py \
  dimos/benchmark/evaluation/test_policy_runtime.py \
  dimos/benchmark/libero_pro \
  dimos/manipulation/test_grasp_execution.py \
  dimos/perception/test_grounded_segmentation.py -q

Run the frozen development panel:

set -a; source .env; set +a
CONFIGURE_SYSTEM=false \
LCM_DEFAULT_URL='udpm://239.255.76.67:7791?ttl=0' \
uv run python -m dimos.benchmark.libero_pro.autoresearch \
  --panel dimos/benchmark/libero_pro/cases/autoresearch/dev-panel.json \
  --output /tmp/dimos-libero-autoresearch \
  --json

Validated locally:

• 52 focused tests pass
• Ruff passes on every Python file changed by this PR
• the retained development run scores 1/4 with zero failures
• an unchanged confirmation run reproduces the same native object success

Current local mypy also reports the inherited ImageContent(mimeType=...) call in code_policy_server.py; this PR does not modify that line.

AI assistance

Codex with GPT-5.6 performed the autonomous experiment loop, implementation, transcript/video audits, tests, and PR drafting. The author directed the benchmark contract and general-capability constraints and reviewed the resulting design interactively.

Checklist

• I have read and approved the CLA.

[TomCC7]

Superseded by the linear stack #3581 → #3582 → #3583, rooted on #3439. Closing this draft to keep the review boundaries clear; its branch is preserved.