Skip to content

feat(settings-advisor): enhance scan detection and adopt traits-driven model selection - #20

Merged
kwrkb merged 4 commits into
masterfrom
feature/settings-advisor-improvements
Sep 12, 2026
Merged

kwrkb merged 4 commits into
masterfrom
feature/settings-advisor-improvements

Conversation

@kwrkb

@kwrkb kwrkb commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Summary

Enhance workspace scan detection accuracy and transition model recommendation logic to a data-driven (traits-based) design for settings-advisor.

1. Enhanced Workspace Scanning

  • Expanded CI/CD Detection: Added support for GitLab CI (.gitlab-ci.yml, .gitlab/ci/), CircleCI (.circleci), Bitbucket (bitbucket-pipelines.yml), and Azure Pipelines (azure-pipelines.yml) alongside GitHub Actions.
  • Language Support Expansion: Added detection for Dart (.dart), Swift (.swift), Vue (.vue), Svelte (.svelte), PHP (.php), Scala (.scala), Zig (.zig), Lua (.lua), SQL (.sql), and Terraform/OpenTofu (.tf).
  • Build/Artifact Directory Skipping: Added .next, .nuxt, out, target, .dart_tool, and Pods to skipDirs.
  • Safe .env Detection: Excluded dummy template files (.env.example, .env.sample, .env.template, .env.dist, .env.test) to prevent false-positive sandbox recommendations.
  • Production Config Directory Hierarchy Detection: Detects production configuration files located in directories containing prod/production tokens (e.g. config/prod/app.json).

2. Traits-Driven Model Selection & models.json Update

  • Updated models.json: Upgraded Flash tier models to Gemini 3.8 Flash series.
  • Eliminated Hardcoded Model Names: Replaced direct model name strings (Claude Sonnet 4.6 (Thinking), GPT-OSS 120B (Medium)) in main.go with dynamic matching against model traits (instruction-following, quota-independent, etc.).

3. Documentation & Testing

  • Updated and synchronized README.md and README_ja.md.
  • Added unit and regression tests in main_test.go (all tests passing).
  • Documented architectural decision in LESSONS.md and updated PLAN.md.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 33656d9ca8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

".html": true,
".css": true,
".sh": true,
".dart": true,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Rebuild the bundled binaries for the changed scanner

Installed users execute the committed native binary through gemini-extension.json, but this commit changes main.go without updating any of the three native binaries. I confirmed that the bundled Linux binary still reports a Dart-only workspace as 0 lines, treats .env.example as a real environment file, and misses GitLab CI, so the documented scanning enhancements are not delivered by a normal plugin installation. Rebuild and commit every distributed platform binary.

AGENTS.md reference: AGENTS.md:L46-L48

Useful? React with 👍 / 👎.

Comment on lines +75 to +80
"target": true,
".next": true,
".nuxt": true,
"out": true,
".dart_tool": true,
"Pods": true,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not skip the requested scan root

When the explicitly supplied repository path itself is named one of the newly added entries such as target, out, or Pods, filepath.Walk invokes this branch for the root and immediately returns filepath.SkipDir, producing an empty scan. The skip check should apply only to descendant directories (for example, when path != root) so arbitrary valid repository paths remain scannable, with a path-boundary regression test.

AGENTS.md reference: AGENTS.md:L42-L42

Useful? React with 👍 / 👎.

Comment thread settings-advisor/src/main.go Outdated
Comment on lines +255 to +257
if strings.Contains(slashPath, ".github/workflows") ||
strings.Contains(slashPath, ".gitlab/ci") ||
strings.Contains(slashPath, ".circleci") {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Match CI directories on path-component boundaries

For workspaces containing paths such as .circleci-disabled/job.yml or .gitlab/circle/config.yml, these substring checks set HasCI even though neither path is one of the supported CI directories, causing an unwarranted proceed-in-sandbox recommendation. Compare normalized path components, or require a separator boundary after .circleci and .gitlab/ci, and cover near-matching directory names in tests.

AGENTS.md reference: AGENTS.md:L42-L42

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 51a6e34ce7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread settings-advisor/src/main.go Outdated
Comment on lines +206 to +208
dir := filepath.Dir(slashPath)
for _, part := range strings.Split(dir, "/") {
if hasProdToken(strings.ToLower(part)) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Limit production matching to paths beneath the scan root

When an absolute scan root has an ancestor named prod or production (for example, /mnt/production/repos/app), slashPath includes that out-of-workspace component, so any JSON/YAML/TOML file marks HasProdConfig and causes an unwarranted strict permission recommendation. Derive the directory from a path relative to root before checking its components, and cover this ancestor-boundary case.

AGENTS.md reference: AGENTS.md:L41-L42

Useful? React with 👍 / 👎.

Address Codex review findings on PR #20.

- skipDirs no longer skips the explicitly requested scan root, so a
  repository named `target` / `out` / `Pods` is still scannable.
- CI and production-config detection now run off root-relative path
  components instead of substring matches on the absolute path. This
  fixes two faces of the same bug: `.circleci-disabled/` and
  `.gitlab/circle/` no longer set HasCI, and an out-of-workspace
  ancestor such as `/mnt/production/repos/app` or `~/.circleci/...`
  no longer sets HasProdConfig / HasCI.
- Regression tests for all three cases (each verified to fail against
  the pre-fix scanner), plus a no-regression test covering real CI
  directory layouts.
- Rebuilt the three distributed binaries with the pinned go1.26.5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@kwrkb

kwrkb commented Sep 11, 2026

Copy link
Copy Markdown
Owner Author

Codex レビューの4件すべてに対応しました(986d6a6)。

P1 — Rebuild the bundled binaries: 対応済みです。指摘のレビュー対象コミット 33656d9 の直後、51a6e34 で3プラットフォーム分を再ビルド済みでした。決定論ビルド(GOTOOLCHAIN=go1.26.5 ./build.sh settings-advisor)を回して git diff が空になることを確認しています。今回の修正分も同じ手順で再ビルドしてコミットしました。

P2 — Do not skip the requested scan root: skipDirs の判定に path != root を追加。target / out / Pods / build / dist という名前のリポジトリを直接指定しても走査されます。

P2 — Match CI directories on path-component boundaries / P2 — Limit production matching to paths beneath the scan root: この2件は同一バグの別断面だったため、まとめて直しました。isCIPath も絶対パスに対する部分文字列一致だったので、prod 側と同じ「スキャンルート外の祖先ディレクトリ名が混入する」問題を抱えています(例: ~/.circleci/repos/app/foo.yml で HasCI が立つ)。個別に直すと prod 側だけ root 相対になり CI 側の祖先混入が残るため、relDirComponents(root, path) で root 相対のパス成分を一度だけ求め、CI 判定(.circleci は成分完全一致、.github+workflows / .gitlab+ci は隣接成分一致)と prod 判定の両方をそこから駆動する形に統一しました。

テスト: 以下を追加し、いずれも修正前のスキャナに対して FAIL することを確認済みです(通過してしまう見せかけのテストでないことの確認)。

  • TestScanWorkspaceRootNamedSkipDir — ルート名が target/out/Pods/build/dist
  • TestScanWorkspaceCINearMissDirs — .circleci-disabled/, .gitlab/circle/, .github/workflows-archive/, .gitlab/ci-templates/, docs/circleci.md
  • TestScanWorkspaceAncestorOutsideRoot — production / prod / .circleci を祖先に持つスキャンルート(実ディレクトリを t.TempDir() 配下に作成)

加えて TestScanWorkspaceCIComponentMatch で、成分単位比較にしても正規の CI ディレクトリ配置を取りこぼさないことを担保しています。

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 986d6a645c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread settings-advisor/src/main.go Outdated
Comment on lines +247 to +249
suffix := strings.TrimPrefix(fileName, ".env.")
switch suffix {
case "example", "sample", "template", "dist", "test", "defaults", "schema":

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Ignore qualified environment templates

When a workspace contains a common qualified template such as .env.production.example or .env.local.sample, suffix becomes production.example or local.sample, so this exact-value switch misses it and returns true. This incorrectly sets HasEnv and recommends enabling the sandbox despite the documented template exclusion; recognize terminal template markers while preserving actual files such as .env.production, and cover qualified templates in the filter test.

Useful? React with 👍 / 👎.

`isEnvFile` compared the whole suffix against an exact set, so a common
qualified template such as `.env.production.example` or `.env.local.sample`
fell through and set HasEnv, recommending the sandbox despite the
documented template exclusion.

Match the terminal dot-separated token instead. This catches qualified
templates while keeping files that hold real values, including
`.env.production` and `.env.test.local`.

Tests cover both forms; both new tests were verified to fail against the
previous implementation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@kwrkb

kwrkb commented Sep 12, 2026

Copy link
Copy Markdown
Owner Author

P2「Ignore qualified environment templates」に対応しました(c0ea9d9)。

指摘どおり isEnvFile は接尾辞を完全一致で判定していたため、.env.production.example は suffix == "production.example" となってどのケースにも当たらず true を返していました。末尾のドット区切りトークンで判定する方式に変更しています。

実値を持つファイルは残す必要があるので、境界はこう置きました。

判定
.env.production / .env.local / .env.test.local / .env.production.local env ファイル(true)
.env.production.example / .env.local.sample / .env.staging.template / .env.production.dist テンプレート(false)

.env.test.local を true にしているのは、Next.js / Vite 系で実値を入れて使う実ファイルであり、末尾トークン local はテンプレートマーカーではないためです。

テストは TestIsEnvFileQualifiedTemplates(18ケースの表駆動)と TestScanWorkspaceQualifiedEnvTemplates(修飾テンプレートのみのワークスペースで HasEnv=false)を追加し、どちらも修正前の実装に対して FAIL することを確認済みです(.env.production.example, .env.local.sample, .env.staging.template, .env.production.dist の4件が true として検出されていました)。

バイナリも .go-version 固定の go1.26.5 で再ビルド済み、CI 全7ジョブ green です。

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c0ea9d9573

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

"sample": true,
"template": true,
"dist": true,
"test": true,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Treat .env.test as an actual environment file

When a repository stores test-environment variables in .env.test, this marker makes isEnvFile return false, so HasEnv remains false and the sandbox recommendation is suppressed. .env.test commonly denotes an environment-specific configuration rather than a template; only unambiguous template suffixes such as example, sample, and template should be excluded.

Useful? React with 👍 / 👎.

".tf": true,
}

var skipDirs = map[string]bool{

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Skip Terraform and Swift dependency caches

When scanning an initialized Terraform or Swift workspace, .terraform/modules and SwiftPM's .build/checkouts contain dependency source with the newly counted .tf and .swift extensions, but neither directory is in this skip map. Those vendored files are therefore included in TotalLines and can push a small workspace into the mid or heavy tier despite the advertised package/build exclusion; add these standard cache directories and path regression tests.

AGENTS.md reference: AGENTS.md:L40-L42

Useful? React with 👍 / 👎.

@kwrkb
kwrkb merged commit 05446c4 into master Sep 12, 2026
7 checks passed
@kwrkb
kwrkb deleted the feature/settings-advisor-improvements branch September 12, 2026 02:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant