Skip to content

feat(bench): configure and compare candidate component switches - #404

Merged
T-Auto merged 4 commits into
mainfrom
work/benchmark-switches-296
Oct 25, 2026
Merged

T-Auto merged 4 commits into
mainfrom
work/benchmark-switches-296

Conversation

@Haoming9527

@Haoming9527 Haoming9527 commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Benchmark setup can now select candidate components through repeatable --switch flags for hide-official, PTC, panel, discovery and shell. It preserves inherited literal settings, refuses conflicting local values without --force, and refuses dynamic or unsafe patch shapes. Baseline profiles receive no component-switch writes. Dry-run prints the plan without launching the host or writing profiles.

Real setup verifies composed configuration and requires a successful bounded headless readiness turn before reporting success; READINESS_BLOCKED and other readiness failures point to #207 before any sweep starts. This turn may call the configured model. Effective switch facts are recorded in summaries and sealed arms.cand.switches, displayed in reports and the offline panel, and compared between candidate runs. Different candidate sets fail comparison with exit 2; baseline's empty set is expected. Unevaluated values remain unknown.

Fixes #296. Stacked on #402 (work/resolve-294-model-matrix) to keep this change under the implementation-size limit; #402 includes #399 and #381. Includes the reviewed #401 workflow change so PRs targeting feature branches run CI. Merge prerequisites first. #303 administrator enforcement and #207 Desktop validation remain open; this PR claims neither.

Validation:

  • New behavior was red against the unwired setup/comparison paths. Independent review added red regressions for dynamic config preservation, legacy activation flags and unknown metadata; all now pass.
  • Ten component-switch checks pass, including the actual official host config merger, candidate-only writes, readiness failures, CLI exit 2 and a rendered DOM panel assertion. The seal suite checks schema-valid top-level arm switch records.
  • All 20 deterministic benchmark suites pass on the final combined branch; no provider/model request was made during verification.
  • Workflow/ruleset lint, source-size and whitespace checks pass. Bilingual README and Unreleased entries included.
  • Current-head PR CI at 8481d91 passed every executed check, including Windows, Linux, macOS, Node-floor and CI required. The all-target trigger ran successfully against the stacked feature-branch base. Conditional smoke was skipped as designed.

Thatgfsj
Thatgfsj previously approved these changes Oct 24, 2026

@Thatgfsj Thatgfsj left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

复审无误,批准合并

@T-Auto
T-Auto changed the base branch from work/resolve-294-model-matrix to main October 25, 2026 04:55
@T-Auto
T-Auto dismissed Thatgfsj’s stale review October 25, 2026 04:55

The base branch was changed.

@T-Auto
T-Auto merged commit 0a98673 into main Oct 25, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Benchmark] 计划:支持组件开关矩阵(hide-official / ptc / discovery 等)

3 participants