Recursive Self-Improvement Infrastructure for Physical Agents

Technical Report · Website · Documentation · Discord

English · 简体中文

What · Why · v1.1.0 · Quick Start · Examples · Control Modes · Benchmark · Robot Skills · Documentation

GOSIM Shenzhen 2026 Spotlight · October 16–17 · Meet us in Shenzhen

---

What is PhyAgentOS?

PhyAgentOS is a Recursive Self-Improvement (RSI) framework for embodied agents. It connects cognitive planning, Physical Execution, and task verification in a runtime-wide feedback loop, improving Skills and Lessons from verified experience for future tasks.

Why PhyAgentOS?

Capability	What it enables
One framework, multiple control modes	Combine general-model reasoning, action models, or hybrid control through the capabilities exposed by an installed Skill.
Verified task outcomes	Evaluate goals against observations and execution facts, with bounded replanning when recovery is needed.
Experience that improves future tasks	Accumulate reusable workflows and scoped Lessons from verified experience, with guarded promotion and revision history.
Reusable physical capabilities	Package environment-specific tools and runtimes as versioned Skills, keeping cognitive planning separate from robot integration.

What's new in v1.1.0

Provider management, Skill Runtime and governed interactions. This release adds:

• Provider management: centralized configuration, connection tests, authentication, model discovery, saved model selection, and per-session overrides.
• More model connections: direct OpenAI Responses API support for tools, multimodal input, reasoning effort, and Forge verification; Requesty, Cheaper Inference, and API Route gateways.
• Skill Runtime updates: directory_tar_gz Node packages with bundled dependencies and validated internal relative symbolic links; profile startup timeouts, required_tools overrides, and startup-hook context for LIBERO 0.3.4.
• Governed interaction foundation (opt-in): connect AgentTask and Session to multi-step decisions, independent root verification, and one deduplicated experience episode through frozen bindings, a persistent Runner, and a Supervisor. See scope and enablement.
• Reliability improvements: preserve valid cron jobs, handle Dora status and profile environments, improve archive and image-path handling, and retain conversation history on model failures. Cancellation blocks new execution intents, late decisions are discarded, and uncertain operations retain guards until reconciliation without automatic retransmission.
• Guides and validation: expanded regression and build checks, bilingual documentation, Piper and LIBERO-10 quickstarts, and clearer benchmark scope and results.

See the full changelog and changes since v1.0.0.

Quickstart · How to get started

Start with a model-backed CLI conversation, then connect a robot or simulator Skill below. Prerequisites: Python 3.11 or 3.12, Git, and an API key for your model provider.

1. Install and initialize

git clone --branch v1.1.0 https://github.com/PhyAgentOS/PhyAgentOS-core.git
cd PhyAgentOS-core
python -m venv .venv
source .venv/bin/activate
python -m pip install -e .
paos onboard

On Windows PowerShell, replace the activation command with .venv\Scripts\Activate.ps1.

2. Connect a model

Configure credentials with hidden input, then select the default:

paos provider configure openrouter
paos provider use openrouter --model anthropic/claude-sonnet-4

See the provider CLI guide for Docker/secret input, process overrides and session commands (/provider, /model, /effort, /status). Session switches affect subsequent turns only and leave running tasks and other sessions unchanged.

Full configuration reference (v1.1.0)

The configuration file is serialized in camelCase; snake_case keys are also accepted.

{
  "agents": {
    "defaults": {
      "workspace": "~/.PhyAgentOS/workspace",
      "model": "openrouter/openai/gpt-4o-mini",
      "provider": "openrouter"
    },
    "verification": {
      "serviceEnabled": true,
      "evidenceRetention": "failed",
      "maxReplansPerEpisode": 2,
      "maxVerifierCallsPerRun": 50
    },
    "evolution": {
      "enabled": true,
      "scope": "verified_forge_lineage",
      "promotionMode": "guarded_auto",
      "minSuccessfulEpisodes": 3,
      "minLessonEpisodes": 3,
      "maxLessonsPerSkill": 8,
      "maxEvolutionCallsPerRun": 20
    }
  },
  "providers": {
    "openrouter": {
      "apiKey": "YOUR_API_KEY"
    }
  },
  "forge": {
    "requestTimeoutS": 10,
    "pollIntervalS": 0.5,
    "executionTimeoutS": 300,
    "evidence": {
      "requiredImageSources": ["front"],
      "captureTimeoutS": 5,
      "postCaptureTimeoutS": 5,
      "connectionTimeoutS": 2,
      "maxArtifactBytes": 8388608,
      "associationQuality": "best_effort"
    }
  },
  "resourceRegistry": {
    "url": "https://paos-resource-manager.dev.x-era.com"
  }
}

The front source is only an example. resourceRegistry.url selects a generic package registry; it may be empty when all artifacts are installed from local bundles or a supplied static index. PAOS connects only to the Gateway URL in the manifest of the explicitly started, healthy Skill Runtime. It never starts or downloads a concrete Skill merely because the Agent starts.

3. Run your first request

paos status
paos agent -m "Hello! Introduce yourself."
paos agent

You should receive a model response; paos agent opens an interactive conversation. This checks the core Agent and provider connection. Physical execution additionally needs a running Skill Runtime.

Need another provider or deployment option? See the user manual · Docker.

One framework, three control modes

System 1 / System 2 follow the supplied figure’s naming for general-model control and action-model control, respectively.

These describe integration patterns; model and environment availability depends on the installed Physical Execution Skill. Core does not bundle model weights, simulator assets, or robot drivers.

Benchmark

Reported task success rates on LIBERO-Long, RoboDojo, and RoboTwin. Purple bars show PhyAgentOS (PAOS); muted bars show public-leaderboard references.

Evaluation scope: LIBERO uses libero-10 (labeled LIBERO-Long in the figures). RoboDojo and RoboTwin each use 10 randomly sampled tasks; sample counts vary for some models. Public-leaderboard references may use different protocols.

PhyAgentOS configuration	LIBERO-10 ↑	RoboDojo ↑	RoboTwin ↑
GPT-6 + π0.5	100.00%	23.30%	76.70%
DeepSeek 4.1 Flash + π0.5	97.22%	26.70%	77.80%
GLM 5.3 Flash + π0.5	100.00%	20.00%	66.70%
Gemini Robotic ER2 + π0.5	—	—	80.00%
Kimi-K3 + π0.5	—	—	66.70%
Gemini 3.8 Flash + π0.5	—	—	66.70%
DeepSeek 4.1 Flash	82.00%	8.30%	—
GPT-6	—	8.30%	—
GLM 5.3 Flash	—	2.78%	—
Kimi-K3	60.00%	0.00%	—

English translations of the latest maintainer-supplied figures. See full results, time and token usage, and evaluation context. “—” means not reported. To reproduce the LIBERO evaluation, follow the LIBERO quickstart.

Connect a robot or simulator

A Physical Execution Skill packages a workflow and its runtime requirements. Install the Skill for your environment, start one of its named profiles, then ask the Agent to use it.

1. Install Dora for managed runtimes

v1.1.0 retains the Dora CLI 0.4.1 (dora-message 0.7.0) compatibility baseline. On Linux/macOS:

curl --proto '=https' --tlsv1.2 -LsSf \
  https://github.com/dora-rs/dora/releases/download/v0.4.1/dora-cli-installer.sh | sh
dora --version

Dora is needed for paos skill start; the CLI conversation above does not require it. Windows and Cargo instructions.

2. Find and install a Skill

For registry downloads, set resourceRegistry.url in your config or export PAOS_RESOURCE_REGISTRY_URL to the registry supplied by your deployment. Then run:

paos skill search
paos skill install <skill-name> --version <version>
paos skill inspect <skill-name>

Replace <skill-name> and <version> with a search result. For an independently obtained local bundle:

paos skill install /path/to/skill-bundle.tar.gz --local

3. Start the runtime and submit a task

Replace <profile> with a profile declared by the installed Skill; follow its setup instructions for models, assets, and hardware.

paos skill start <skill-name> --profile <profile>
paos skill status <skill-name>
paos agent -m "Inspect the active Skill and Physical Execution capabilities, then report which tasks are available."

Once the runtime is healthy, describe a task supported by that Skill in paos agent. Starting the Agent does not automatically download or start a runtime.

Useful runtime commands

paos skill list
paos skill logs <skill-name>
paos skill stop <skill-name>
paos gateway

paos gateway runs configured message channels and background services. For readiness checks and troubleshooting, see the operations guide.

Each Physical Execution Skill bundle declares its workflow document, required Tool IDs, named runtime profiles, and exact platform/architecture Node locks. Each locked archive has an exact SHA-256 and contains either one named root-level executable (executable_tar_gz) or one root directory named after the entrypoint that holds the executable and its runtime tree (directory_tar_gz). For Registry Node downloads, the verified Skill lock supplies the digest when the Registry omits that duplicate field, and the exact size is resolved before the download begins; installation records and verifies the extracted binary hash. python scripts/package_skill.py <bundle-dir> --output-dir <directory> creates a deterministic bundle for publication. The PhyAgentOS source and release packages do not bundle concrete Physical Execution Skills, Physical Execution nodes, models, or simulation assets; obtain only the Skills needed for a deployment and install them explicitly. The integration development guide documents Bundle layout, local validation, immutable publication order, and Registry acceptance.

Quickstart examples

Platform	Coverage	Guide
Piper robot arm	Skill installation, CAN setup, hardware startup, and tool readiness	Piper quickstart (Chinese)
LIBERO-10 simulation benchmark	Skill/Node installation, π0.5 weights, model API setup, and a 50-episode run (π0.5-only or GPT-6 supervised)	LIBERO quickstart (Chinese)

Documentation

I want to…	Start here
Install, configure, and run PhyAgentOS	User manual
Build a Skill or connect an environment	Integration guide
Understand the Physical Execution API	Physical Execution Tool API
Develop and test Core	Developer manual
Inspect benchmark values and provenance	Benchmark
Browse all documentation	Documentation index

News

2026-10-10 · GOSIM Shenzhen 2026: PhyAgentOS has been selected for Spotlight and will present in the Open Source Robotics track. Read the announcement and event details.

Version	Date	Update
v1.1.0	2026-10-09	Centralized provider management, direct OpenAI Responses and new gateways, Skill Runtime packaging and startup improvements, and an opt-in governed interaction foundation.
v1.0.0	2026-08-30	Initial stable release of PhyAgentOS.
v0.2.3	2026-08-27	Physical Execution Skills can be installed and managed independently, activated into immutable AgentTask bindings, and used through governed Query, Action, and Session Tool API lifecycles with recovery and version-scoped experience.
v0.2.2	2026-08-21	Unified Physical Execution execution on the Query/Action Tool API and added AgentTask aggregation, a verifiable Skill Runtime, Resource Registry integration, and the move-arm-by-ee Skill while retaining Agent verification and evolution.

See the full changelog.

Contributing & community

Contributions are welcome: report a bug, improve a Skill, add an environment integration, or help with documentation. Start with the developer manual and open an issue or pull request.

Development checks

python -m pip install -e ".[dev]"
pytest
ruff check PhyAgentOS tests
python -m compileall -q PhyAgentOS tests

Discord · X · Bilibili · LinkedIn · Xiaohongshu

Citation & acknowledgements

If PhyAgentOS helps your research, please cite our technical report.

@article{liu2026phyagentos,
  title={PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution},
  author={Liu, Yang and Chen, Weixing and Song, Xinshuai and Pu, Tao and Mo, Siwen and Bai, Yongjie and Chen, Zihao and Sun, Qianran and Zhong, Liruo and Shen, Ying and others},
  journal={arXiv preprint arXiv:2607.16636},
  year={2026}
}

Thanks to the maintainers of MuJoCo, ROS, the open benchmark suites, and every contributor to PhyAgentOS.

---

Jointly developed by Sun Yat-sen University HCP Lab, Peng Cheng Laboratory, and X-Era Lab.

     

_{MIT License · Copyright © 2025–2026 PhyAgentOS}