Skip to content

governance(reviews): adaptive multi-reviewer convergence, authority and repository learning #780

Description

@qnbs

CURRENT AUTHORITATIVE ROLE — adaptive multi-reviewer convergence, authority and repository learning — 2026-10-01

This issue is now the provider-neutral integration owner for WorldScript Studio's advisory review ecosystem and its durable learning loop. The original #779/CodeRabbit work remains historical input, but this owner is no longer CodeRabbit-centric.

Current review sensors include, when available:

Codex
Cubic
CodeRabbit
DeepSource
CodeScene
CodeAnt
Sourcery
plus deterministic CI/security/test gates

Priority / execution mode

STATE = ACTIVE_SUPPORT_LANE
PRIMARY_RELEASE_LANE = #445 → v1.30.0
MAY_PREEMPT_R15 = NO, unless a fresh validated P0/P1/release invariant requires it
CURRENT_LIVE_EXEMPLAR = PR #930 / Gate 3A
BLOCKING_AUTHORITY_CHANGE = NO
AUTOMATIC_PROVIDER_PROMOTION = FORBIDDEN

Work that can be safely pulled forward now:

  • classify all available reviewer findings on exact heads;
  • preserve the one-coherent-correction-wave rule;
  • encode validated recurrence-prevention in tests/types/contracts/CI/instructions where naturally in scope;
  • keep reviewer availability/quota/config incidents distinct from clean review;
  • keep source-truth for advisory-vs-blocking authority accurate;
  • curate the reviewer capability/authority matrix and exact-head evidence model;
  • use current R-15 PRs as evidence, without widening their semantic scope.

Broader workflow/governance mutations remain bounded follow-ups or #927 work unless they are required to correct a current release-critical defect.

Provider-neutral reviewer contract

For every reviewer/provider record at least:

PROVIDER
SIGNAL_TYPE              = inline review | review body | commit status | deterministic gate
CURRENT_HEAD_SHA
STATE                    = CLEAN | FINDINGS | PENDING | UNAVAILABLE | QUOTA_LIMITED | CONFIG_INCIDENT | STALE_HEAD
BLOCKING_ROLE             = DIRECT_REQUIRED | TRANSITIVE_REQUIRED | POLICY_ONLY | ADVISORY_ONLY | UNKNOWN_EXTERNAL
CONFIG_AUTHORITY          = repo | dashboard | external | mixed/unknown
MAY_MUTATE_BRANCH         = false by default
MAY_APPROVE/MERGE         = false unless separately authorized
LAST_VERIFIED

A provider becoming available again after a quota/month reset restores coverage, not authority.

Adaptive learning loop

Validated nontrivial findings should follow:

observe
→ independently validate
→ classify severity/scope
→ fix in one coherent wave
→ exact-head prove
→ capture recurrence-prevention at the lowest enforceable layer
→ verify no governance drift
→ continue

Preferred durable learning hierarchy:

type/API invariant
> regression/fault/property test
> deterministic CI/static guard
> normative contract/schema
> AGENTS/instruction rule
> prose-only reminder

Do not accumulate cargo-cult rules for one-off nits.

Existing specialist owners — consume, do not duplicate

Current DeepSource truth

The accidental JavaScript-dashboard incident on main is historical/resolved. JavaScript remains disabled pending separate post-v1.30 admission. On PR #930, DeepSource Rust findings are real current-diff review evidence and must be handled in the normal full epoch.

No reviewer becomes required merely because comments or monthly quota became available again.


Current purpose — 2026-09-23

PR #779 established the repository's reviewer-governance foundation, but its terminal convergence intentionally left two classes of residual work:

  1. bounded reviewer-governance hardening that should not have expanded an already-green PR; and
  2. autonomous review-remediation convergence control, after chore(governance): codify reviewer policy and config #779 reached 14 signed source commits through repeated reviewer-driven correction waves.

A later post-Visual/CodeRabbit planning pass adds a third durable responsibility that belongs here rather than in a new competing issue:

  1. CodeRabbit capability / GitHub-App permission / bounded-Autofix governance, including explicit entitlement gates, least-privilege permission admission, Canary A/B, rollback, and the exact human/admin boundary for any GitHub setting or app-permission change.

This issue is therefore the durable integration owner for reviewer-convergence policy and CodeRabbit reviewer/remediation governance. It must not become an unbounded hardening PR itself.


A. Autonomous review-convergence contract

Required operating rule for autonomous agents:

initial implementation
→ exact-head CI/reviews
→ collect the whole current review epoch
→ ONE normal bundled root-cause correction wave
→ exact-head revalidation
→ terminal disposition / merge when safe

A later source-mutation wave is allowed only when the exact current head contains a demonstrable CURRENT_REAL correctness, security, data-integrity, merge-safety, or other release-invariant defect. Do not reject a real current defect merely to preserve an arbitrary wave count, but do not reopen a green exact head for speculative defense-in-depth.

After an exact head is green:

  • historical/outdated findings are dispositioned, not reimplemented;
  • provider quota/rate-limit/unavailability is never treated as clean-review evidence;
  • reviewer silence is not REVIEWED_CLEAN;
  • do not manually retrigger quota-exhausted providers merely to seek more comments;
  • collect inline threads, top-level comments, and full review bodies before another mutation;
  • branch-protection/admin changes remain explicit human/admin actions.

Coordinate with #550 for measurable review/CI churn. This issue owns when to stop mutating; #550 owns pipeline efficiency.


B. Deferred reviewer-governance hardening from #779

Revalidate and close only bounded current residuals:

  • register the separate Reviewer Governance Trust Boundary as a required protected status only if repository branch/ruleset policy chooses to require it and the exact status identity is proven;
  • harden trusted setup-action admission against execution-affecting env / unexpected inputs with canonical step-shape validation rather than an endless blacklist;
  • strengthen positive CodeRabbit path-filter coverage so a syntactically valid filter set cannot accidentally exclude all governed code;
  • validate required path-instruction semantics beyond mere non-empty strings where a stable machine contract is justified;
  • close durable provider-state key aliases such as direct providerState / providerStatus forms if the registry contract requires it;
  • expose reviewed commit SHA / exact-head match in reviewer evidence where materially useful;
  • keep .coderabbit.yaml, docs/REVIEWER-GOVERNANCE.md, config/reviewer-registry.json, reviewer-status tooling, and the trusted reviewer-governance workflow mutually consistent.

These are bounded follow-ups, not retroactive blockers for merged PR #779.


C. Branch-protection / admin boundary

At the #779 audit point, repository ruleset/protection state was not fully readable through the available GitHub App administration permission.

Therefore use explicit states:

ADMIN_STATE_VERIFIED
ADMIN_STATE_UNVERIFIED
ADMIN_CHANGE_REQUIRED
NO_ADMIN_CHANGE_REQUIRED

Rules:

  • repository source must not claim branch protection has been live-verified when it has not;
  • required-status changes are external/admin work;
  • no automation may bypass or silently modify branch protection;
  • document the desired protected-status contract first;
  • stop only at the exact authorized human/admin action when one is genuinely required;
  • after the maintainer performs it, re-fetch and verify the resulting state before relying on it.

C1. Registry policy vs effective merge authority

A follow-up DEV.to comment on “When AI Reviewers Become a CI Problem” identified a real trust-accounting ambiguity:

A durable registry field such as blockingAuthority: false is only trustworthy if external branch-protection/ruleset configuration and vendor-side status behavior cannot silently contradict it.

This issue owns that reconciliation. Do not create a second reviewer-governance parent.

Current semantic gap

config/reviewer-registry.json is durable repository policy. It is not, by itself, proof of live GitHub merge enforcement.

In particular, blockingAuthority: true must not be read as “this provider is necessarily a direct required status context”. Live enforcement can be:

DIRECT_REQUIRED
  provider/check itself is a required protected context

TRANSITIVE_REQUIRED
  provider/tool runs inside or gates a required aggregate such as ✅ CI Success

POLICY_REQUIRED_BUT_NOT_PLATFORM_BOUND
  repository process treats it as mandatory, but branch protection does not mechanically require it

ADVISORY_ONLY
  signal/reviewer may report findings but cannot mechanically block merge

UNKNOWN_EXTERNAL
  dashboard/service behavior is not currently verified

Example from current evidence: OSV is enforced transitively through the required CI aggregate, while CodeQL runs separately and is not one of the three required branch-protection contexts currently proven by #875. Therefore the existing boolean is too coarse to serve as a live enforcement proof without an accompanying reconciliation step.

Required reconciliation contract

Before claiming reviewer/governance authority is terminal, derive the effective authority graph from live state:

  1. fetch current branch protection / rulesets with maintainer-authenticated read-only gh;
  2. enumerate every required status context and its producing workflow/event;
  3. for required aggregate jobs, map their required needs closure and identify which deterministic tools/providers gate that aggregate;
  4. compare the resulting direct/transitive enforcement map with config/reviewer-registry.json;
  5. for configurationOwner: dashboard / live-service, verify any vendor setting that can change GitHub check behavior or review disposition where accessible;
  6. classify unverifiable external settings as UNKNOWN_EXTERNAL, never as clean;
  7. record drift as a governance finding before changing either repository policy or platform settings.

The comparison must distinguish desired policy from effective enforcement. A mismatch is not automatically resolved by editing the registry to match the platform; first decide which side is wrong.

Tooling direction

The offline pnpm run reviewers:check remains deterministic/network-free and must not gain privileged GitHub credentials.

Instead, extend or complement the existing read-only operational tooling (for example reviewers:status) with an explicit authority-reconciliation mode that can run from a maintainer-authenticated host context and report, without mutating:

REGISTRY_DECLARED_ROLE
DIRECT_REQUIRED
TRANSITIVE_REQUIRED
ADVISORY_ONLY
UNKNOWN_EXTERNAL
DRIFT

Do not make PR CI depend on admin credentials merely to read branch protection. If GitHub later exposes a least-privilege, base-owned, non-secret way to prove the same state, that may be reconsidered separately.

Vendor-side drift

A semantic reviewer remains non-blocking only if all relevant live mechanisms agree:

  • it is not a required status context;
  • it is not part of the required aggregate dependency graph;
  • it does not have an authorized request changes / merge-gating mode that repository policy treats as authoritative;
  • no vendor dashboard setting silently changes its role;
  • no ruleset/branch-protection change elevates its status without corresponding registry/docs reconciliation.

Likewise a deterministic security tool marked as blocking must have an explicit direct or transitive path explaining how it blocks.

pull_request_target invariant

The DEV.to feedback also correctly emphasizes the supply-chain risk of privileged pull_request_target.

WorldScript's base-owned Reviewer Governance Trust Boundary may use pull_request_target only under the strict invariant already intended here:

base-owned workflow/code only
+ PR head fetched/read strictly as data
+ no execution of PR scripts/dependencies
+ no checkout/use of untrusted PR code under secrets/write credentials
+ least-privilege permissions

Any future pull_request_target workflow that executes PR-controlled code, installs PR-controlled dependencies, or exposes secrets/write credentials is a trust-boundary defect, not an implementation convenience.

Acceptance delta

Add these terminal requirements to this issue:

  • registry semantics explicitly distinguish durable policy from live enforcement;
  • every blockingAuthority: true entry has a documented direct/transitive/policy-only enforcement path;
  • every semantic reviewer intended non-blocking is reconciled against live required contexts / aggregate dependencies and relevant verified vendor behavior;
  • unresolved external/dashboard state is represented as unknown, never silently assumed;
  • the operational authority reconciliation is read-only and cannot mutate protection/settings;
  • no privileged pull_request_target path executes untrusted PR code or dependencies;
  • governance(protection): prove required-check binding, workflow origin and bypass policy before further hardening #875 live protection evidence is consumed rather than duplicated or contradicted.

D. CodeRabbit authority and least-privilege capability model

The durable role remains reviewer-first:

CodeRabbit
  = independent semantic reviewer
  + bounded remediation agent only for explicitly eligible findings

Claude / Codex / maintainer
  = primary implementers for architecture, security, persistence,
    recovery, migration, authority and ambiguous remediation

GitHub CI / repository governance
  = deterministic verifier

Preserve unless an explicit later architecture decision changes them:

  • config/reviewer-registry.json: mayMutateBranch = false as the default governance stance;
  • CodeRabbit must not become merge authority;
  • no auto-merge;
  • no branch-protection bypass;
  • no direct trust exemption for CodeRabbit-authored changes;
  • no weakening of Verified Signatures or commit-attribution requirements;
  • no reviewer self-approval as sole validation.

Before changing permissions, inventory live CodeRabbit product capability and GitHub App permissions from first-party/current sources.

Build a capability→permission matrix:

CAPABILITY
REQUIRED_PERMISSION
CURRENT_GRANT
ACTUALLY_NEEDED
SECURITY_IMPACT
MINIMUM_SCOPE
ENABLE / KEEP / DENY
ROLLBACK

Do not grant write/admin scopes merely because the provider offers a feature.


E. CodeRabbit schema/config + false-positive governance

Revalidate the live CodeRabbit schema before assuming historical keys still exist.

Required outcomes where still current:

  • remove unsupported/stale fake Autofix / fix-CI / merge-conflict YAML keys rather than pretending they enable a product feature;
  • do not invent replacement keys;
  • preserve reviewer-first settings and high-value path instructions;
  • keep request_changes_workflow: false and fail_commit_status: false unless a separately authorized policy change proves otherwise;
  • preserve disabled automatic docstring/unit-test/simplify mutations unless separately justified;
  • no broad auto-label/auto-assignment expansion without evidence;
  • maintain positive governed-code path-filter coverage;
  • maintain semantic path-instruction tests rather than brittle exact prose tests.

Repository-policy findings must follow authority order:

scoped AGENTS.md / normative repo policy
→ machine-enforced checker + tests
→ canonical reviewer/CI governance docs
→ .coderabbit.yaml adapter
→ source examples
→ historical docs

Do not mutate valid source merely to satisfy a model-inferred convention.

The recurring QNBS-v3 false-positive class is a mandatory regression target: tagged forms allowed by canonical checker/policy stay valid; [Grund / Impact / Kreativer Mehrwert] is not universally mandatory; obvious regression tests do not need a marker merely because they are new/important.


F. Autofix entitlement and execution-queue gate

Never infer Autofix availability from an old prompt or config key.

Before any mutation experiment, live-verify:

  • actual current CodeRabbit plan/entitlement;
  • available Autofix commands/modes;
  • whether stacked/separate PR delivery exists;
  • whether Autofix processes all unresolved CodeRabbit findings;
  • permissions required for ordinary-source mutation;
  • permissions required for .github/workflows/** mutation.

If Autofix is unavailable on the current plan/product:

AUTOFIX_ENTITLEMENT_BLOCKED
→ keep reviewer-quality/governance improvements
→ do not grant workflow-write for Autofix
→ do not stall the engineering program

If Autofix is available, treat unresolved CodeRabbit threads as an execution queue.

Before triggering:

  1. fetch every current CodeRabbit thread;
  2. validate each against repository authority;
  3. resolve/disposition false positives, stale findings, duplicates, out-of-scope items, privileged/ineligible findings, and low-value suggestions;
  4. leave unresolved only the exact approved execution set;
  5. re-fetch;
  6. require:
unresolved CodeRabbit threads
==
approved Autofix execution set

Otherwise do not run Autofix.


G. Autofix eligibility boundary

Eligible only after triage

Small deterministic ordinary-source/test fixes such as:

  • straightforward assertion correction;
  • obvious null/undefined correctness fix;
  • narrow CSS selector/specificity correction;
  • simple type error with unambiguous semantics;
  • small isolated refactor with no authority/security/persistence implication;
  • small visual regression correction;
  • narrow YAML/action syntax/input correction with unambiguous behavior.

Privileged / stacked-or-separate-PR only

Treat as elevated:

  • .github/workflows/**;
  • workflow-policy and reviewer-governance trust files;
  • Dependabot config;
  • release/signing/provenance/deployment automation;
  • permission blocks;
  • secret-consuming workflows;
  • pull_request_target;
  • artifact publication;
  • security configuration;
  • CODEOWNERS/ruleset-adjacent governance.

No direct-to-main and no auto-merge.

Never CodeRabbit-Autofix authority in this program

Route to Claude/Codex/maintainer:

  • core(project-boundary): finish current canonical no-loss persistence/import/snapshot closure #553 canonical persistence/storage authority;
  • preserve-first data-integrity paths;
  • filesystem authority;
  • CAS/locking/generation fencing;
  • encryption/key ownership/key rotation;
  • recovery and destructive migrations;
  • native/Tauri/Rust authority boundaries;
  • non-trivial security/CSP/secret-handling architecture;
  • PWA multi-tab consistency;
  • release-signing trust architecture;
  • destructive data operations;
  • ambiguous findings with multiple valid architectures;
  • anything that can weaken tests, gates, branch protection, signing, reviewer independence or persistence authority.

H. Canary A — ordinary-source Autofix before workflow permission

Only if live entitlement is available.

Use a naturally occurring, harmless, reversible, non-privileged CodeRabbit finding.

Prefer the safest isolated/stacked PR mode.

Before trigger, enforce the execution-queue equality above.

The canary must avoid workflow, security, persistence, migration, native authority, release/signing and destructive-operation paths.

Inspect:

  • exact generated diff and changed-file set;
  • commit author/committer/message;
  • GitHub verification;
  • Verified Signatures;
  • commit attribution;
  • whether any non-approved file/finding was touched;
  • CI and independent review;
  • resulting-main CI/CD + CodeQL after merge.

Canary A passes only if scope, identity, signatures, attribution, CI, review independence and normal merge governance all pass.

If Canary A fails:

  • keep CodeRabbit reviewer-only;
  • avoid CodeRabbit mutation;
  • preserve evidence;
  • do not grant workflow-write;
  • continue the engineering program.

I. Workflows: read/write permission hard gate

Do not request this permission merely because workflow Autofix exists.

Immediately before any request, require:

#510 source-side trusted-execution disposition = ACCEPTABLE
no known PR-tree self-weakening path relevant to workflow mutation = OPEN
trusted workflow-policy/reviewer boundary = PASS
PR-size/write-scoped authority = PROVEN or explicitly ADVISORY_ONLY
required-status/admin state = VERIFIED or exact human action identified
no untrusted PR code executes under privileged pull_request_target path
normal signature/attribution/workflow-policy gates remain authoritative
Canary A = PASS
Autofix entitlement = AVAILABLE
workflow mutation use case = CONFIRMED
stacked/separate PR path = CONFIRMED
permission rollback path = DOCUMENTED

Then stop at the exact human boundary and report:

CODERABBIT WORKFLOW-WRITE PRECONDITIONS PASSED

SAFE NEXT MANUAL ACTION:
Approve CodeRabbit GitHub App permission:
Workflows: read/write

The permission grant is a manual maintainer/admin action. Automation must not assume or self-grant it.

If any source-side trust item is unresolved, do not request the permission.


J. Canary B — workflow-write

Only after explicit maintainer confirmation that the exact required workflow permission is granted.

Use a naturally occurring, harmless, reversible, low-risk workflow finding.

Requirements:

  • .github/workflows/** is the only privileged area intentionally exercised;
  • stacked/separate PR mode;
  • no direct-to-main;
  • no auto-merge;
  • never manufacture a vulnerability or break a release path just to create a canary;
  • do not use release/signing/deployment/security-critical semantic changes.

Inspect especially:

  • permissions:;
  • pull_request_target;
  • action SHA pins;
  • persist-credentials;
  • secret usage;
  • shell commands;
  • artifact publication;
  • release/deploy behavior;
  • workflow-policy and reviewer-governance compatibility;
  • signature/attribution compatibility.

If no safe naturally occurring workflow finding exists, keep workflow mutation unproven and disabled until a legitimate canary appears.

Canary B passes only if the workflow diff is bounded, independently reviewable, least-privilege, trust-boundary safe, signature/attribution clean, normal CI green, and resulting-main CI/CD + CodeQL green.

On failure:

  • stop workflow Autofix;
  • roll back/reduce the permission where practical;
  • keep ordinary CodeRabbit review;
  • preserve evidence;
  • continue the engineering program.

K. Production operating mode after proven canaries

Default remains reviewer-first.

Bounded Autofix is allowed only for a fully triaged approved execution queue and small deterministic eligible scope.

Direct branch mutation is never the default merely because it is faster.

Workflow Autofix stays stacked/separate-PR only unless a later explicit governance change authorizes otherwise.

Use at most one Autofix execution per validated root-cause/review wave unless a new independent real defect appears.

After mutation:

allow CI + normal incremental review
→ collect complete epoch
→ classify
→ do not reflexively Autofix every new nit

No recursive reviewer↔remediator loop.


L. Auditability / rollback / terminal evidence

For every CodeRabbit mutation experiment record in PR evidence as appropriate:

trigger command/mode
validated unresolved-thread set
eligible findings
false positives resolved before trigger
generated PR/commit SHA
changed files
signature result
attribution result
CI result
independent review result
resulting-main result
disposition

Do not create a permanently tracked live-state ledger that becomes stale immediately.

Keep explicit rollback for any permission expansion.

Do not claim a capability that was not actually exercised.


Relationships / ownership

Do not create another CodeRabbit parent issue unless a genuinely separate durable ownership domain emerges.


Acceptance criteria

  • Autonomous-agent instructions encode whole-epoch collection, bounded root-cause remediation and a real stop condition.
  • A green exact head is not reopened for speculative hardening without a current real defect.
  • Provider quota/rate-limit/unavailability remain distinct from reviewed-clean evidence.
  • Deferred chore(governance): codify reviewer policy and config #779 technical hardening is implemented only as bounded independently reviewable work.
  • Reviewer-status evidence can identify reviewed commit/exact-head relationship where admitted.
  • CodeRabbit config validation has positive governed-code path coverage and stable semantic path-instruction validation.
  • Live CodeRabbit schema/product/plan are revalidated before configuration or permission decisions.
  • Unsupported stale mutating config keys are removed rather than treated as real capabilities.
  • Repository policy hierarchy and QNBS-v3 false-positive prevention are encoded/tested.
  • A live CodeRabbit capability→permission matrix exists for the intended features.
  • No unnecessary write/admin permission is granted.
  • Autofix entitlement is explicitly classified as AVAILABLE or BLOCKED.
  • Canary A proves or rejects ordinary-source mutation under normal signature/attribution/CI/review governance.
  • Workflows: read/write is never requested before ci: write-scoped governance jobs (pr-size, workflow-policy) enforce from PR-controlled code, not a trusted ref #510/trust/admin gates + Canary A pass.
  • Any workflow permission grant is an explicit maintainer/admin action with a rollback path.
  • Canary B proves or rejects workflow mutation in stacked/separate PR mode without weakening least privilege, SHA pinning, pull_request_target safety, signing or attribution.
  • A failed or unavailable Autofix capability falls back safely to reviewer-only operation and does not stall the engineering program.
  • CodeRabbit-authored changes never receive a trust exemption or self-review-only admission.
  • Any desired required-status/branch-protection change is documented and performed only through authorized admin action.
  • ci(efficiency): reduce redundant cost and latency without weakening required quality/security gates #550 receives measurable review/CI churn evidence rather than this issue inventing a second CI-efficiency authority.
  • No implementation PR for this issue repeats the chore(governance): codify reviewer policy and config #779 correction cascade.
  • The issue remains open for any explicitly deferred reviewer-governance residual that is not terminally satisfied.

Priority

P2 governance / autonomous-convergence safety. This is important trust/integration infrastructure, but it is not automatically a patch-release blocker. Promote only if live evidence proves a current fail-open reviewer/governance defect invalidates merge/release evidence or a required pre-release capability depends on it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions