Skip to content

ci: add staged coverage gates, baseline comparison, and PR coverage template - #6

Merged
kusutori merged 1 commit into
mainfrom
codex/add-code-coverage-step-to-ci-pipeline
May 6, 2026
Merged

kusutori merged 1 commit into
mainfrom
codex/add-code-coverage-step-to-ci-pipeline

Conversation

@kusutori

@kusutori kusutori commented May 6, 2026

Copy link
Copy Markdown
Owner

Motivation

  • Introduce automated coverage enforcement into CI with staged thresholds to improve test quality over time.
  • Prevent coverage regressions by comparing PR coverage to the base branch.
  • Require higher coverage for critical internal packages to reduce risk in key code paths.

Description

  • Update CI workflow .github/workflows/ci.yml to generate coverage with go test ./... -coverprofile=coverage.out and go tool cover -func=coverage.out, fetch the base branch and compute a base coverage using a temporary worktree, and invoke the coverage gate script with COVERAGE_STAGE and BASELINE_COVERAGE environment variables.
  • Add scripts/coverage_gate.sh which implements staged thresholds (stage1=60, stage2=70, final=80), parses total coverage from the coverage profile, enforces per-package stricter thresholds for internal/config and internal/install (>=85%), and fails if current coverage is below the provided BASELINE_COVERAGE.
  • Add a PR template .github/pull_request_template.md with coverage fields (before/after, affected packages) and a checklist that requires running go test ./... -coverprofile=coverage.out, go tool cover -func=coverage.out, no regression versus base, and critical package thresholds.

Testing

  • Ran go test ./... -coverprofile=coverage.out and go tool cover -func=coverage.out to produce coverage artifacts, which completed and printed per-package results.
  • Executed the gate with COVERAGE_STAGE=final scripts/coverage_gate.sh coverage.out, which correctly parsed total coverage (38.4%) and failed because it is below the final threshold (80%).
  • The gate also reported per-package coverage for internal/config (71.3%) and internal/install (21.6%) and failed those per-package checks as expected, validating the stricter-package logic.

Codex Task

@kusutori
kusutori merged commit 49fec77 into main May 6, 2026
0 of 2 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d0c88232ed

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/ci.yml
run: scripts/coverage_gate.sh coverage.out
env:
COVERAGE_STAGE: final
BASELINE_COVERAGE: ${{ env.BASELINE_COVERAGE }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Propagate baseline coverage from runtime env

The baseline regression check is effectively disabled on pull requests because this step sets BASELINE_COVERAGE from ${{ env.BASELINE_COVERAGE }}, but that context only includes workflow/job/step env keys, not values written at runtime via $GITHUB_ENV in the previous step. As a result, BASELINE_COVERAGE is passed as empty and scripts/coverage_gate.sh skips the head < baseline guard (if [[ -n "$BASELINE_COVERAGE" ]]), so coverage regressions can merge unnoticed.

Useful? React with 👍 / 👎.

Comment thread .github/workflows/ci.yml
- name: Coverage gate
run: scripts/coverage_gate.sh coverage.out
env:
COVERAGE_STAGE: final

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Start CI coverage gate at a staged threshold

This hard-codes COVERAGE_STAGE to final, which makes the staged rollout logic in scripts/coverage_gate.sh (stage1|stage2|final) unreachable and immediately enforces the strictest threshold on every run. That defeats the intended staged adoption and will cause CI to fail continuously on branches that have not yet reached final targets, blocking normal PR flow instead of gradually tightening coverage gates.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant