Skip to content

Fix rollout lifecycle, recovery, and validation - #12

Merged
AltmanD merged 2 commits into
mainfrom
integrate/dimweaker-rollout-fixes-20261103
Nov 3, 2026
Merged

AltmanD merged 2 commits into
mainfrom
integrate/dimweaker-rollout-fixes-20261103

Conversation

@AltmanD

@AltmanD AltmanD commented Nov 3, 2026

Copy link
Copy Markdown
Owner

将原作者 Dimweaker 的 7 个commit 适配到当前 main。

  • 修复 rollout 关闭、任务取消和重启恢复,避免清理中断与重复执行。
  • 加强生成文本、token 和采样参数的一致性校验。
  • 改善 Web 日志异常处理、部署路径和原生示例上下文配置。

AltmanD and others added 2 commits November 3, 2026 15:33
Manually port Dimweaker's seven fixes from
fix/rollout-lifecycle-and-validation-20261030 onto main at 230b171.
The source history has no common ancestor with main.

Join real executor Tasks before closing runtime resources; retain durable
group outcomes across restarts without replaying interrupted Runners; hold
an OS lock on the service run directory. Keep log failures from blocking
Web recovery, normalize target paths lexically, and raise the native
example context budget to 32768.

Require complete serving text/token agreement and bind every loaded slot
to the requested sampling digest. Include the necessary numeric sampling
canonicalization and cleanup admission checks for Docker and Sandbox,
while preserving main's embedded path and optional math distillation.

Source commits: 673659b, 6e2e7f9, a8f7e1d, 03caa72, 64b92e2,
f08e6ba, 9814460 (source base ebb1efc).

Validation: 350 CPU tests passed, 8 optional tests skipped. Example smoke,
training dry run, shell/JavaScript syntax and public-tree audit passed.
Changed Python files pass Ruff; main has five existing E402 findings in
tests/test_consistency_torch.py. No live training acceptance is claimed.

Co-authored-by: Dimweaker <443668214@qq.com>
Shield the group gather so run_group owns cancellation propagation.
Without this, cancelling the parent first cancels every slot through
gather, then a faster slot triggers another cancellation that interrupts
a slower sibling's asynchronous finally block.

Cancel each child once and wait for the original gather and all slots
before allowing service shutdown to close the runtime. Add a regression
that uses the real run_group with two slots of different cleanup speeds.

Validation: the new regression fails before this fix and passes after it;
the full CPU suite passes with 351 tests and 8 optional skips.
@AltmanD
AltmanD merged commit f710793 into main Nov 3, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant