Skip to content

Reuse equivalent main CI evidence at release without dropping pinned integration coverage #175

Closed

Description

@kikashy

The verify / "CI at the tag" job in release.yml calls ci.yml, which already runs on main.

Allow reuse of successful equivalent exact-commit evidence. Preserve both Runner Linux/macOS jobs and Release contents validation under the exact release toolchain. Pinned Runtime, Gateway and old-Runner integration inputs must match; named real-companion tests must actually run and pass, not skip.

Keep release packaging, native archive smoke checks, current tag/provenance/notes checks, protected publication, checksums, attestations and draft/published asset verification. Do not count a unit-only or different-toolchain run as the release's integration proof.

Coordinate the common evidence contract and test fixtures with Devkit #37. Each repository owns its adoption and configuration; do not copy an unverified "latest green" lookup.

Required evidence gate

  • Inventory equivalent checks and retain intentional OS/toolchain, dependency-pin, integration, tag-context and artifact differences. The same commit alone does not prove equivalent execution inputs.
  • Reuse only a completed successful run from the expected workflow identity in this repository, bound to the exact release commit, trusted main-branch event/ref, workflow/check definition and required job/matrix coverage. A PR head, synthetic merge, another workflow, mutable status name or tree-equivalent commit is insufficient.
  • Define deterministic run/attempt selection: do not search past a newer failed, canceled or running applicable attempt to find an older green result. Missing, inaccessible, partial, stale or incompatible evidence must block reuse. Document a bounded fresh-validation route; unavailable evidence never becomes success.
  • Verify all required jobs and coverage, with explicit permitted skips. Retain evidence run IDs/attempts, source/workflow/input identities, job results and the reuse decision in release logs. Account for pagination and reruns.
  • Keep fresh checks for mutable external state where existing policy requires them, including advisory freshness and current tag/protection state. Do not cache away these decisions.
  • Test accepted exact-commit evidence and refusals for wrong SHA/repository/event/workflow, missing jobs/matrix legs, unexpected skips, failed/canceled/running attempts, changed inputs, unavailable evidence and tag movement. Removing the admission safeguard must make its intended test fail.
  • Compare executed checks and observed duration before/after; do not promise savings from unmeasured totals. Update release documentation and preserve required branch-protection statuses, review and publication requirements.

Future maintenance only: do not alter an in-flight release or start implementation in another session's active lane. Scheduled for v0.25.0 by maintainer request.

Activity

  1. added this to the v0.25.0 milestone on Oct 25, 2026
  2. added 2 commits that reference this issue on Oct 26, 2026
  3. kikashy commented on Oct 26, 2026

    @kikashy
    ContributorAuthor

    Shipped in #182 (530afd5): the release reuses main's exact-commit CI evidence at the tag through the kit's v0.11.0 gate, keeps both Runner jobs and Release contents, carries the pinned integration inputs and the named real-companion tests' PASS lines in the evidence, and fails closed. Capture first runs on the next push to main; the gate first runs live at the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions