Skip to content

Propose checks from a rule on save, confirmed, edited or rejected by its author - #2669

Merged
pournasserian merged 4 commits into
v2from
2650-rule-compile
Oct 26, 2026
Merged

pournasserian merged 4 commits into
v2from
2650-rule-compile

Conversation

@pournasserian

@pournasserian pournasserian commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Closes #2650

What

The second slice of F-RUL (#2419), RUL-04 with MDL-03: saving a rule proposes checks that enforce it, and the author confirms, edits or rejects each one. It builds on the rule book of #2643 and the job handler conformance suite now in v2.

  • Core (FluentCMS.Core.Rules):
    • RuleCompiler asks the model on a new rules profile.
      • The instructions describe the built-in types a rule can compile to: terms, pattern, length, range, tone, claims and reading level.
      • The rule's text and scope go in the prompt's data sections, never in its instructions (SEC-03, RUL-07).
      • The answer is read in a fixed shape: up to three checks, each with its settings and at least one passing and one failing example. A proposal takes the rule's severity and scope.
      • A proposal of a type that was not offered, or that does not validate against the installed check types, is left out.
      • No model, a failed call, an unreadable answer, or no usable proposal leaves the version advisory, with a message that says why.
    • RuleCompileJob (rules.compile) runs the compile as the check engine, on behalf of the person who saved the rule. EngineJobs lists it for the check engine. The payload only names the version; whether the job is still that version's compile comes from what is stored. If its last attempt throws, it records the version advisory, so a version never stays compiling.
    • IRuleChecks:
      • reads a version's compile and its proposals
      • proposes again, which marks the undecided proposals replaced
      • confirms a proposal, as proposed or edited, under its proposed name or another
      • rejects a proposal
      • for the job only: reads the version it compiles, and records what it found
  • Infrastructure:
    • Saving a rule version (create or revise) schedules its compile in the same unit of work: a job when the rules profile is configured, and otherwise an advisory compile at once.
    • Tables RuleCompilations and RuleCheckProposals, with migrations for SQLite and PostgreSQL; a proposal's definition is jsonb on PostgreSQL.
    • Confirming creates the check and activates it once its examples get their verdicts. The check, its check.created and check.activated events, the proposal's decision and rule.check_confirmed commit together. The examples run before the unit of work, so no model call happens inside it.
    • CheckDirectory.ListActiveAsync leaves out a check confirmed from a rule while the rule is not active at the version the check was proposed for. So the check run, check_draft and the inline-or-job decision all follow the rule's state.
  • Docs: CONTRIBUTING.md on the compile, and ADR-0012 now lists proposing checks among the check engine's actions.

Why

RUL-04 turns a rule's prose into checks the author can trust: the system proposes, a person decides (D17). The next slice labels rules enforced or advisory and gives agents the rule book with its compiled checks, which needs to know which checks serve which rule and version.

Choices worth a look:

  • Compiling is a job, as the check engine. A model call can take seconds, so saving a rule never waits for one. The job runs as the check engine, since proposing checks is the engine's work, and is charged to the rule's author. A person's calls are uncapped, as for the check run. No new engine was added.
  • A check serves the rule version it was confirmed for. It guards content only while that version is active. A draft rule's checks wait for its set to publish it, a retirement stops them, a revert brings them back, and publishing a revision hands over to the checks confirmed for it. The cost is that a revision's checks are confirmed again, even for a small edit; that keeps "what runs" equal to "what the author confirmed for this text".
  • Confirming also activates. Activation is what runs a check's examples, and a person confirms (D17), so one step does both. The examples must get their verdicts, or nothing is stored.
  • Proposal numbers are never reused. Proposing again marks undecided proposals replaced rather than deleting them, so a number the author read always names the same proposal. It first moves the compile to its new job, which waits for a compile that is recording at the same time, so that compile's proposals are replaced too.
  • Check names come from the rule, as <rule slug>-<number>, not from the model. The author can give another name when confirming.
  • Not in this PR: enforced and advisory labels, the rule book tool and resource and rule hints (RUL-05, RUL-06, MCP-02, MCP-03), flagging instruction-like rule text (RUL-07), and proposing a rule from the desk (RUL-08).

Tests

Unit:

  • A voice and style rule compiles to a terms check, with the rule's severity and scope and its examples. The request goes to the rules profile on the author's behalf, with the rule's text and scope only in the data sections.
  • No model is configured: the rule is advisory and no call is made.
  • A failed call leaves the rule advisory, with the reason.
  • An answer that cannot be read leaves the rule advisory. An answer in a code fence is still read.
  • A proposal of an unknown or unoffered type, without a failing example, or with settings that are not an object is left out.
  • A rule no check can enforce is advisory.
  • At most three checks are proposed.
  • The rule compile runs as the check engine, and rules.compile passes the job handler conformance suite.
  • A compile payload whose version is not a whole number fails as payload_invalid.
  • A compile that throws is retried, and its last attempt leaves the version advisory with a message.

Integration, each on SQLite and PostgreSQL:

  • Without a model, a saved rule is advisory with the message, and no compile is queued.
  • The compile job, run as the check engine for the author, proposes the two checks. The model call's system message does not hold the rule's text, which sits inside the user message's data delimiters. rule.compiled names the model and the prompt hash.
  • A confirmed check is created and active, with its events. It does not run while the rule is a draft. Once the set publishes the rule, "best in class" fails it; once the rule is retired, it does not run.
  • A revision's confirmed check takes over from the active version's when the revision is published.
  • An edited confirmation creates the edited check under another name. A confirmation whose examples fail, or whose name is taken, stores nothing.
  • A confirmed or rejected proposal cannot be decided again, and an unknown number is refused.
  • Proposing again marks the undecided proposals replaced and numbers new ones after them. The compile it replaced records nothing.
  • An unreadable answer leaves the version advisory.
  • An agent and an engine are refused confirming, rejecting and proposing again. A person and an agent are refused reading or recording a compile as the job does.

The full suite ran locally against a postgres:17 container as well as SQLite, and markdownlint passes. No new package. Tests use a fake model provider: no real model calls, no keys, no network.

Checklist

  • Linked to an issue a maintainer has confirmed
  • Targets v2
  • Tests or docs updated where behaviour changed

@pournasserian pournasserian added this to the M1 Governance core milestone Oct 26, 2026
@pournasserian pournasserian added type:feature New capability v2 FluentCMS v2 (v2 branch) area:rules Rule book (RUL) labels Oct 26, 2026
@pournasserian
pournasserian merged commit 74ba131 into v2 Oct 26, 2026
2 checks passed
@pournasserian
pournasserian deleted the 2650-rule-compile branch October 26, 2026 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:rules Rule book (RUL) type:feature New capability v2 FluentCMS v2 (v2 branch)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Propose checks from a rule on save, confirmed, edited or rejected by its author

1 participant