Skip to content

Default to OpenRouter provider; add arm-motion video recording (motion.mp4) - #2

Merged
Daniel-rmc merged 2 commits into
Daniel-rmc:harness_v1from
liuzhenO:add_harness
Oct 27, 2026
Merged

Daniel-rmc merged 2 commits into
Daniel-rmc:harness_v1from
liuzhenO:add_harness

Conversation

@liuzhenO

Copy link
Copy Markdown

Summary

  • Default provider → OpenRouter (https://openrouter.ai/api/v1), default model openai/gpt-6-astra. No API keys are included in this change — credentials are still supplied at runtime via environment variable or the UI, as before.
  • New MotionRecorder (src/maniloop/recording/motion.py): snapshots MjData inside the physics loop on the simulation clock, renders both camera views offscreen (osmesa) in a dedicated thread, and streams frames to ffmpeg to produce motion.mp4 (320 px side panel, 20 fps, queue backpressure, zero dropped frames). Under controlled timing the physics freezes during API inference, so the video contains only actual arm motion — no idle waiting.
  • Runner integration: recorder lifecycle wired into runner.py (create on episode start, snapshot per control step, finalize on terminate/close).

Verification

  • Recorded full episodes on the built-in MuJoCo tasks; motion.mp4 plays back both camera views with no dropped frames.
  • Audited the diff and working tree for credentials: none present.

Made with Cursor

liuzhen and others added 2 commits October 22, 2026 02:00
- 默认服务商改为 OpenRouter(base_url: openrouter.ai/api/v1),
  默认模型 openai/gpt-6-astra;不包含任何 API key,密钥仍由
  环境变量或界面输入在运行期提供
- 新增 recording/motion.py:物理循环内按仿真时钟快照,
  独立线程离屏渲染双视角并经 ffmpeg 流式合成 motion.mp4;
  controlled 时序下 API 推理期间物理冻结,视频只含机械臂实际运动
- runner 接入录制生命周期(启动创建、步进快照、终止/关闭收尾)

Co-authored-by: Cursor <cursoragent@cursor.com>
- 新增 maniloop/harness 包(抽象自 Goal2Skill 上层大脑,剔除任务特化):
  types/memory/planner/verifier/reflector/bridge/router/vlm/agent
- 双时标回路:VLM 规划为语义子任务(终态可视觉验证),执行器逐步执行;
  L0 运动规则门控 + L1 执行器自报 PHASE 行 + L2 仅在宣称完成时 VLM 确认,
  验证成本与子任务数同量级而非与决策步数同量级
- 失败经独立 reflector 诊断(retry/adjust-param/replan),
  前缀保留式重规划;尝试/重规划/回退次数均有硬上限
- HarnessAgent 实现标准 Agent 协议,runner 主循环零改动;
  策略切换/暂停单步/密钥刷新路径均已兼容
- UI 策略下拉新增「分层大脑 Harness」,复用云端 LLM 凭证与模型选择
- 不包含任何 API key;角色模型默认跟随当前连接模型,可经 harness_roles 覆盖

Co-authored-by: Cursor <cursoragent@cursor.com>
@Daniel-rmc
Daniel-rmc changed the base branch from main to harness_v1 October 27, 2026 14:17
@Daniel-rmc
Daniel-rmc merged commit 3c7befd into Daniel-rmc:harness_v1 Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants