Skip to content

Add CI automation: benchmarks, AI labeler, path labeler - #5

Merged
jeremy merged 5 commits into
mainfrom
ci-automation-gaps
Mar 5, 2026
Merged

jeremy merged 5 commits into
mainfrom
ci-automation-gaps

Conversation

@jeremy

@jeremy jeremy commented Mar 5, 2026

Copy link
Copy Markdown
Member

Summary

  • Benchmarks CI — runs on push to main with benchstat regression detection (≥20% = failure), cached baselines, step summary output
  • AI PR labeler — classifies PRs as bug/enhancement/documentation for release.yml changelog; detects breaking changes to exported Go API surface
  • Path-based labeler — auto-labels PRs by changed paths (ci, tests, docs, deps, skills, seed, actions, prompts)
  • Editor package — new editor.Open(initialContent) API for $EDITOR integration, used by CLIs that need interactive text composition
  • Seed Makefile — adds check-toolchain, test-coverage, and coverage targets

Closes the CI gap between the seed templates and this library repo. The labeler workflows will self-exercise on this PR.

Also pushed directly to main (trivial):

  • CODEQL_EXTRACTOR_GO_BUILD_TRACING env on CodeQL build step
  • Rubric-audit skill scorecard totals updated to match current RUBRIC.md (26/16/11/29=82)

Test plan

  • make check passes locally
  • AI labeler fires on this PR and classifies it
  • Path-based labeler applies labels (expect: ci, prompts)
  • After merge: benchmarks job runs on main push

Copilot AI review requested due to automatic review settings March 5, 2026 03:01

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds GitHub Actions automation to reduce CI gaps vs the seed templates by introducing benchmark regression checks and automated PR labeling (path-based and AI-based) to support release tooling.

Changes:

  • Add a benchmarks job to the existing test workflow (run on pushes to main) with baseline caching and benchstat comparison.
  • Add a path-based PR labeler workflow and label rules.
  • Add an AI-driven PR classifier (bug/enhancement/documentation) plus an AI-based breaking-change detector for exported Go API diffs, backed by prompt files.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
.github/workflows/test.yml Adds a benchmarks job with caching + benchstat comparison/regression detection.
.github/workflows/labeler.yml Adds a workflow to apply labels based on .github/labeler.yml.
.github/workflows/ai-labeler.yml Adds AI PR classification + breaking-change detection workflows.
.github/prompts/classify-pr.prompt.yml Defines the classification prompt used by the AI labeler.
.github/prompts/detect-breaking.prompt.yml Defines the breaking-change detection prompt and JSON schema.
.github/labeler.yml Defines path-to-label mapping rules for the labeler workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/test.yml Outdated
Comment thread .github/workflows/test.yml
Comment thread .github/workflows/ai-labeler.yml
Comment thread .github/workflows/test.yml Outdated
Comment thread .github/workflows/test.yml Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b8ba80a6d7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/test.yml Outdated
jeremy added 3 commits March 5, 2026 00:25
Runs on push to main when Go files change. Uses benchstat to compare
against cached baseline; >20% regression emits ::error:: and fails.
Results posted to step summary and uploaded as artifact.
Two-job workflow: classify labels PRs as bug/enhancement/documentation
for release note categorization; breaking detects exported Go API
surface changes and posts a PR comment when found.
Labels PRs by changed paths: ci, tests, docs, deps, skills, seed,
actions, prompts. Feeds into release.yml changelog categories.
@jeremy
jeremy force-pushed the ci-automation-gaps branch from b8ba80a to d8770b0 Compare March 5, 2026 08:25
Copilot AI review requested due to automatic review settings March 5, 2026 08:44
@jeremy
jeremy force-pushed the ci-automation-gaps branch from d8770b0 to 28330f1 Compare March 5, 2026 08:44

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread seed/Makefile
Comment thread editor/editor.go
Comment thread editor/editor.go Outdated
jeremy added 2 commits March 5, 2026 00:57
check-toolchain detects PATH go vs GOROOT go mismatches (mise
environments); wired as prerequisite to build and test. test-coverage
generates coverage.out and coverage.html. coverage alias auto-opens
the report in a browser.

Document all three in MAKEFILE-CONVENTION.md "Optional (recommended)"
table.
Open(initialContent) launches $EDITOR (default vi) with a temp file,
waits for close, returns the edited text. Supports editors with
arguments (e.g. EDITOR="code --wait") via strings.Fields splitting.
Guards against whitespace-only $EDITOR and empty results.
@jeremy
jeremy force-pushed the ci-automation-gaps branch from 28330f1 to 6c2dc3c Compare March 5, 2026 08:57
@jeremy
jeremy merged commit ba4dc65 into main Mar 5, 2026
15 checks passed
@jeremy
jeremy deleted the ci-automation-gaps branch March 5, 2026 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants