Skip to content

fix(evolve): sync and async are two contracts #817

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Birfy wants to merge 3 commits into
base: fix/804-policies-consolidation
Choose a base branch
from
Open

fix(evolve): sync and async are two contracts #817

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5d076be
merge main into fix/809 after #812
Birfy Nov 6, 2026
e5fdb32
fix(evolve): treat sync and async as two contracts (#809)
Birfy Nov 6, 2026
b840cc1
fix(tests): name max_rollouts for evolve(asynchronous=True) callers
Birfy Nov 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ agentdescent demo # a complete evolution, offline, no AP
agentdescent doctor # which agent CLIs and provider keys this machine has
```

Host / MCP default loop is **Path A** (`doctor` → `plan` → `start` → `apply`); advanced MCP tools need `agentdescent mcp --advanced` — see [docs/mcp-path-a.md](docs/mcp-path-a.md).

`[mcp]` adds the agent-facing MCP server and needs Python ≥ 3.10; the library
itself supports 3.9, where you drop `[mcp]` and drive it with the CLI. Research examples are outside the wheel: to run
`python -m examples.run_demo` or any algorithm port, clone the repo and follow
Expand Down
38 changes: 22 additions & 16 deletions agentdescent/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,29 +6,29 @@
from a shell can be picked up by an agent. The skill file falls back to these
commands on a host with no MCP.

agentdescent demo run one, offline, no key
**Path A** (the default host loop — start here)::

agentdescent doctor what is installed
agentdescent init <path> [--kind ...] [--data cases.jsonl] write a starter spec
agentdescent plan <spec.json> validate + cost, no run
agentdescent evolve <spec.json> [--detach] [--budget USD] start a run
agentdescent status [<run_id>] [--brief] one run or all
agentdescent watch <run_id> follow rounds
agentdescent show <run_id> [--diff] the evolved tree
agentdescent apply <run_id> [--to PATH] [--dry-run] install it, backed up
agentdescent apply --from PATH --to PATH [--dry-run] install an on-disk artifact
agentdescent cancel <run_id>
agentdescent resume <run_id>
agentdescent portfolio [archive] rank layers, reallocate budget
agentdescent exam [archive] --domain module:builder re-judge adoptions with an oracle
agentdescent audit <run_id> --archive PATH trace an adoption's evidence rings
agentdescent doctor what is installed
agentdescent --version which AgentDescent this is
agentdescent install <dsh|claude-code|codex|opencode|grok-bot> wire a host
agentdescent uninstall <dsh|claude-code|codex|opencode|grok-bot> undo `install`
agentdescent algorithms list [--kind K] | show <name> | run … named algorithms for a spec
agentdescent mcp serve over stdio
agentdescent cancel <run_id> / resume <run_id>
agentdescent algorithms list|show|run … named Algorithm cards
agentdescent demo one offline run, no key
agentdescent mcp [--advanced] MCP: Path A by default

**Advanced** (opt-in; MCP needs ``--advanced`` / ``AGENTDESCENT_MCP_ADVANCED=1``)::

agentdescent portfolio / exam / audit / safety / release / …
agentdescent install|uninstall <host> wire a host
agentdescent serve [--port N] read-only run panel

``argparse`` only: the CLI is in the core and the core has no dependencies.
See ``docs/mcp-path-a.md``. ``argparse`` only: the CLI is in the core and the
core has no dependencies.
"""

from __future__ import annotations
Expand Down Expand Up @@ -4205,10 +4205,12 @@ def cmd_mcp(a: argparse.Namespace) -> int:
print(f"agentdescent mcp: streamable-http on http://{host}:{port}/mcp",
file=sys.stderr)
try:
advanced = True if getattr(a, "advanced", False) else None
if transport == "stdio":
serve(store=a.store)
serve(store=a.store, advanced=advanced)
else:
serve(store=a.store, transport=transport, host=host, port=port)
serve(store=a.store, transport=transport, host=host, port=port,
advanced=advanced)
except ImportError as e:
print(f"agentdescent mcp: cannot start the MCP server -- {_mcp_import_failure(e)}",
file=sys.stderr)
Expand Down Expand Up @@ -4982,6 +4984,10 @@ def build_parser() -> argparse.ArgumentParser:
"bound, e.g. already in use")
s.add_argument("--allow-remote", action="store_true",
help="allow a non-loopback --host (the tools can start runs and write files)")
s.add_argument("--advanced", action="store_true",
help="expose audit/meta/platform/experience/… MCP tools "
"(default: Path A only — doctor/plan/start/status/show/apply/…; "
"also AGENTDESCENT_MCP_ADVANCED=1)")
s.set_defaults(fn=cmd_mcp)

sub.metavar = "{" + ",".join(n for n in sub.choices if n not in _INTERNAL_COMMANDS) + "}"
Expand Down
104 changes: 54 additions & 50 deletions agentdescent/evolution.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -865,22 +865,27 @@ def _publish_stable(aggregator) -> None:
pass


#: What `evolve` can honour from a `Policies` bundle today. Everything else
#: What each driver can honour from a `Policies` bundle today. Everything else
#: raises rather than being accepted and ignored -- see
#: `Policies.require_supported`. The set grows as the implementations land.
_WIRED_POLICIES = ("task_sampler", "selection", "proposal", "conflict", "fusion",
"acceptance", "promotion", "staleness", "verifier", "ledger",
"aggregator_factory", "eval_cache", "sandbox_spec", "evaluator",
"executor", "screen")

#: The same bundle, minus the one field the barrier-free loop does not read.
#: `async_evolve`'s worker calls `eng.run` directly; there is no executor seam in
#: it yet, so a supplied one was accepted and then dropped -- the exact outcome
#: `require_supported` exists to make impossible, arriving through a *shared*
#: constant rather than a missing check. It matters more since a supplied
#: executor started working on the synchronous path: flipping
#: `asynchronous=True` would otherwise silently stop honouring it.
_ASYNC_WIRED_POLICIES = tuple(p for p in _WIRED_POLICIES if p != "executor")
#: `Policies.require_supported`. One table so sync and async cannot drift
#: apart without a test noticing (#809). ``executor`` is sync-only: the
#: barrier-free worker calls ``eng.run`` directly and has no executor seam yet.
_POLICY_DRIVER_SUPPORT = {
"evolve": (
"task_sampler", "selection", "proposal", "conflict", "fusion",
"acceptance", "promotion", "staleness", "verifier", "ledger",
"aggregator_factory", "eval_cache", "sandbox_spec", "evaluator",
"executor", "screen",
),
"async_evolve": (
"task_sampler", "selection", "proposal", "conflict", "fusion",
"acceptance", "promotion", "staleness", "verifier", "ledger",
"aggregator_factory", "eval_cache", "sandbox_spec", "evaluator",
"screen",
),
}
_WIRED_POLICIES = _POLICY_DRIVER_SUPPORT["evolve"]
_ASYNC_WIRED_POLICIES = _POLICY_DRIVER_SUPPORT["async_evolve"]


def _propose_via_policy(policy, settled_reader=None):
Expand Down Expand Up @@ -3165,8 +3170,9 @@ def evolve(
Name of the evolving artifact; becomes a filename, so it must match
``[A-Za-z0-9_.-]+``.
rounds:
Number of round barriers to run. Under ``asynchronous=True`` this becomes
a worker-rollout budget of ``rounds * n_workers`` instead.
Number of round barriers to run. Sync-only. ``evolve(asynchronous=True)``
no longer reinterprets this as a rollout budget (#809) — pass
``max_rollouts=`` or call :func:`~agentdescent.async_evolve.async_evolve`.
n_workers:
Workers per round (``>= 1``).
max_concurrency:
Expand Down Expand Up @@ -3231,8 +3237,12 @@ def evolve(
documents what it does and does not fix -- a commit landing mid-rollout
still produces a stale card.
asynchronous, async_ratio:
Delegate to :func:`~agentdescent.async_evolve.async_evolve` -- no round
barrier, with ``async_ratio`` as the staleness lag budget.
**Deprecated bridge** to :func:`~agentdescent.async_evolve.async_evolve`
(#809). Sync and async are two contracts, not a mode flag: prefer
calling ``async_evolve`` directly. When ``asynchronous=True``, both
``max_seconds`` and ``max_rollouts`` are required (no silent
reinterpretation of ``rounds``, no invented 20-second budget).
``async_ratio`` is the async lag budget.
max_seconds:
Wall-clock budget. ``None`` (default) means unbounded; the async path
uses ``20.0`` when unset.
Expand Down Expand Up @@ -3568,12 +3578,10 @@ def evolve(
trajectory_store = _obs.trajectory_store

if asynchronous:
# barrier-free mode: hand the same plug-ins to the async runtime. `rounds`
# becomes a worker-rollout budget (rounds x n_workers) alongside max_seconds.
# Two sync-only knobs have no meaning here and were previously dropped in
# silence, so `parallel=TensorParallel(...)` looked honoured while the run
# was plain DP. Say so instead: the async runtime shards round-robin itself
# and its concurrency *is* `n_workers`.
# Deprecated bridge (#809): sync and async are two contracts, not a mode
# flag. Prefer ``async_evolve`` directly. Validate budgets before the
# deprecation warning so a misconfigured call fails closed without the
# extra noise.
for name, value, why in (
("parallel", parallel,
f"the async runtime shards data-parallel across its own {n_workers} "
Expand All @@ -3593,35 +3601,34 @@ def evolve(
f"evolve(asynchronous=True) ignores {name}=: {why}. Use the "
f"synchronous path if you need it.", RuntimeWarning,
stacklevel=2)
# The two above that are *not* ignored but silently REDEFINED were the
# sharper edge, because nothing said so. Flipping one boolean turned an
# unbounded run into a 20-second one, and a partial artifact with
# `error=None` and a populated `history` is indistinguishable from a
# converged one.
if max_seconds is None:
warnings.warn(
"evolve(asynchronous=True) has no unbounded mode: max_seconds=None "
"becomes 20.0 seconds here, where it means 'no limit' on the "
"synchronous path. Pass max_seconds= explicitly, and check "
"result.stop_reason -- a budget expiry otherwise looks exactly "
"like convergence.", RuntimeWarning, stacklevel=2)
if rounds != 15 and max_rollouts is None: # i.e. the caller chose a value
warnings.warn(
f"evolve(asynchronous=True) has no round barrier, so rounds={rounds} "
f"is reinterpreted as a budget of {rounds * max(1, n_workers)} worker "
"rollouts, and RoundInfo.round becomes a merger-sweep index -- "
"len(result.history) is not comparable with the synchronous path. "
"Pass max_rollouts= to say the budget outright.",
RuntimeWarning, stacklevel=2)
raise ValueError(
"evolve(asynchronous=True) no longer invents max_seconds=20.0 "
"(that made an unbounded sync run look like a 20-second async "
"one). Pass max_seconds= explicitly, or call "
"async_evolve(max_seconds=...).")
if max_rollouts is None:
raise ValueError(
"evolve(asynchronous=True) no longer reinterprets "
f"rounds={rounds} as a budget of "
f"{rounds * max(1, n_workers)} worker rollouts. Pass "
"max_rollouts= for an equal-budget stop (or "
"budget=BudgetSettings(max_rollouts=...)), or call "
"async_evolve(max_iters=...).")
warnings.warn(
"evolve(asynchronous=True) is a deprecated bridge to async_evolve(): "
"sync and async are two contracts (different budgets, different "
"Policies wiring). Call async_evolve(...) directly; see docs/async.md "
"and #809.",
FutureWarning, stacklevel=2)
from .async_evolve import async_evolve
return async_evolve(
tasks, reward, agent=agent, run=run, propose=propose, strategy=strategy,
initial_state=initial_state, blast_radius=blast_radius, artifact_id=artifact_id,
n_workers=n_workers, async_ratio=async_ratio,
resync_on_commit=resync_on_commit,
max_seconds=20.0 if max_seconds is None else max_seconds,
max_iters=(max_rollouts if max_rollouts is not None
else rounds * max(1, n_workers)),
max_seconds=max_seconds,
max_iters=max_rollouts,
max_calls=max_calls, max_tokens=max_tokens, held_out_frac=held_out_frac,
repo_path=repo_path, agg_config=agg_config, staleness_policy=staleness_policy,
aggregator_factory=aggregator_factory, oracle_budget=oracle_budget,
Expand All @@ -3643,9 +3650,6 @@ def evolve(
trajectory_store=trajectory_store, contract=contract,
extra_artifacts=extra_artifacts,
population_islands=population_islands,
# Bags already folded into the flat locals above; forward them so
# the async surface does not look like it dropped the new knobs,
# and a bag-only caller still works if resolve is re-run (#804).
budget=budget, runtime=runtime, observability=observability)

if pipelined_gate:
Expand Down
17 changes: 14 additions & 3 deletions agentdescent/evolvespec.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1305,9 +1305,20 @@ def run(skill: str, task: Task) -> str:
knobs.setdefault("n_workers", max(1, min(default_workers, train)))
if not knobs.get("asynchronous"):
knobs.setdefault("max_concurrency", knobs["n_workers"])
elif "max_seconds" not in knobs:
knobs["max_seconds"] = 600.0
notes.append("asynchronous=True needs max_seconds; defaulted to 600.")
else:
# Async is a second contract (#809). Plan must name budgets in async
# units — never leave rounds to be reinterpreted at evolve() time.
if "max_seconds" not in knobs:
knobs["max_seconds"] = 600.0
notes.append("asynchronous=True needs max_seconds; defaulted to 600.")
if "max_rollouts" not in knobs:
rounds = int(knobs.get("rounds", 15))
workers = int(knobs["n_workers"])
knobs["max_rollouts"] = rounds * max(1, workers)
notes.append(
f"asynchronous=True needs max_rollouts; derived "
f"{knobs['max_rollouts']} from rounds*n_workers at plan time "
f"(rounds is sync-only — pass max_rollouts= to be explicit).")
if "agg_config" in knobs and isinstance(knobs["agg_config"], Mapping):
knobs["agg_config"] = build_agg_config(spec, **knobs["agg_config"])
elif spec.kind == "text" and (spec.agg_config or (ov and ov.agg_config)):
Expand Down
6 changes: 6 additions & 0 deletions agentdescent/integrations/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@ command with the same verbs. A run is an evolution: N workers propose edits in
parallel, a merger keeps the ones that improve held-out reward, and nothing is
written back until the user says so.

**Path A only by default.** The MCP server exposes doctor → plan → start →
status/show → apply (plus cancel/resume, diagnose, list/describe/algorithm_spec).
Do not reach for audit_*, release, platform, experience, or other satellites
unless the user asked and the server was started with `--advanced` /
`AGENTDESCENT_MCP_ADVANCED=1`. See docs/mcp-path-a.md.

## The procedure

1. **`doctor` first.** Report what is missing (worker agent CLI, provider key,
Expand Down
Loading
Loading