Skip to content

fix(evolve): sync and async are two contracts - #817

Open
Birfy wants to merge 3 commits into
fix/804-policies-consolidationfrom
fix/809-sync-async-contracts
Open

Birfy wants to merge 3 commits into
fix/804-policies-consolidationfrom
fix/809-sync-async-contracts

Conversation

@Birfy

@Birfy Birfy commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Closes #809. Sync and async stop pretending to be one evolve(asynchronous=) mode.

  • evolve(asynchronous=True) is a deprecated bridge (FutureWarning) to async_evolve.
  • Fails closed without explicit max_seconds and max_rollouts (no invented 20s budget, no rounds×n_workers reinterpretation).
  • One shared _POLICY_DRIVER_SUPPORT table; test fails if sync/async wiring drifts (executor sync-only).
  • Spec plan derives max_rollouts from rounds*n_workers when missing (named at plan time, not silently at evolve).
  • Docs (async.md, evolution.md) and callers updated; BudgetSettings works on both drivers (evolve: finish Policies consolidation — 57 top-level kwargs past the documented 35 budget #804).

Stacked on #816 (merge #816 first, then retarget this PR to main).

Test plan

  • pytest -q tests/test_sync_async_contracts.py tests/test_provider_and_split_contracts.py tests/test_async_early_stop.py tests/test_budget.py tests/test_token_budget.py
  • tests/test_api_docs.py / changelog fragments

Birfy added 2 commits November 7, 2026 06:29
Deprecate evolve(asynchronous=True) as a bridge to async_evolve: require
explicit max_seconds and max_rollouts (no silent 20s default, no rounds
reinterpretation), share one _POLICY_DRIVER_SUPPORT table, and document
the honest dual entry. Spec derives max_rollouts at plan time when missing.
@Birfy

Birfy commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator Author

PASS

Tip e5fdb32eead3771e2417358e4222d0d03c926f3b — sync/async two contracts (#809). Stacked on #816. risk_gate likely T3 (evolution.py). Bridge fails closed without max_seconds+max_rollouts; prefer async_evolve. Ready for Danyang merge click after #816 lands and CI is green.

@Birfy

Birfy commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator Author

Verdict: PASS

Tip SHA: e5fdb32
Ran: read PR body + tip commit git show vs parent (19 files / #809-focused); full PR-vs-base diff (stacked); linked #809; local pytest -q tests/test_sync_async_contracts.py tests/test_budget.py tests/test_provider_and_split_contracts.py (28 passed); risk_gate.py → T3
Not run: full matrix CI on this box (waiting on GitHub Actions build + test 3.9–3.13)

Summary

Closes #809 acceptance cleanly: evolve(asynchronous=True) is an honest deprecated bridge — fails closed without explicit max_seconds and max_rollouts (no invented 20s, no rounds×n_workers reinterpretation), emits FutureWarning, and forwards to async_evolve. One _POLICY_DRIVER_SUPPORT table + drift test (executor sync-only). Spec plan names max_rollouts at plan time when missing. Docs (async.md, evolution.md, api.md) and callers/tests updated consistently; BudgetSettings covered on both drivers.

Notes

nit (non-blocking)

@Birfy

Birfy commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator Author

Follow-up after required CI on tip e5fdb32eead3771e2417358e4222d0d03c926f3b:

Required checks red — build green; test (3.9)–test (3.13) all FAILURE.

Root cause: callers still using the deprecated bridge without max_rollouts= (and sometimes relying on rounds reinterpretation):

  • tests/test_bench.py — test_staleness_is_measured_on_both_paths, test_the_lag_budget_is_what_made_the_async_row_look_broken, test_a_run_that_discards_most_of_its_evidence_says_so
  • tests/test_fault_matrix.py — all [async] parametrized cases (rounds=12 → ValueError asking for max_rollouts)

Please extend the same migration already done for test_budget / test_async_evolve / test_worker_resilience (pass explicit max_rollouts= or call async_evolve directly), then push a new tip for re-review.

risk_gate remains T3; stacked on #816 — still needs merge #816 → retarget to main → Danyang click after green.

#809/#817 bridge fails closed without explicit max_rollouts. Point the
bench equal-budget matrix and fault_matrix async engine at
rounds×workers so CI matches the documented contract.
@Birfy

Birfy commented Nov 6, 2026

Copy link
Copy Markdown
Collaborator Author

Verdict: PASS

Tip SHA: b840cc1
Ran: patched bench/run.py workload + tests/test_fault_matrix.py async engine to pass explicit max_rollouts= (rounds×workers) for the #809 bridge; pytest -q on the three previously-red test_bench cases + full tests/test_fault_matrix.py (25 passed); pytest -q tests/test_sync_async_contracts.py tests/test_provider_and_split_contracts.py (18 passed); risk_gate.py expected T3 (evolution.py already on PR)
Not run: full GitHub Actions matrix on this tip (build + test 3.9–3.13 still queued after push)

Summary

CI on e5fdb32e failed because evolve(asynchronous=True) now fails closed without max_rollouts. Callers still relied on the old rounds reinterpretation:

  • bench/run.py workload(): async configs now set max_rollouts = rounds * n_workers (equal-budget with the sync matrix).
  • tests/test_fault_matrix.py async engine: max_rollouts=36 (12×3).

Contract unchanged; tests/callers aligned with #809/#817. Stacked on #816; still T3 — do not squash-merge until Danyang clicks after #816 lands and this tip's CI is green.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant