Repository navigation
Store rules as versioned content published through change sets - #2643
Merged
Merged
Conversation
3 tasks done
pournasserian
added a commit
that referenced
this pull request
Oct 26, 2026
…its author (#2669) Closes #2650 ## What The second slice of F-RUL (#2419), RUL-04 with MDL-03: saving a rule proposes checks that enforce it, and the author confirms, edits or rejects each one. It builds on the rule book of #2643 and the job handler conformance suite now in v2. - **Core** (`FluentCMS.Core.Rules`): - `RuleCompiler` asks the model on a new `rules` profile. - The instructions describe the built-in types a rule can compile to: terms, pattern, length, range, tone, claims and reading level. - The rule's text and scope go in the prompt's data sections, never in its instructions (SEC-03, RUL-07). - The answer is read in a fixed shape: up to three checks, each with its settings and at least one passing and one failing example. A proposal takes the rule's severity and scope. - A proposal of a type that was not offered, or that does not validate against the installed check types, is left out. - No model, a failed call, an unreadable answer, or no usable proposal leaves the version advisory, with a message that says why. - `RuleCompileJob` (`rules.compile`) runs the compile as the check engine, on behalf of the person who saved the rule. `EngineJobs` lists it for the check engine. The payload only names the version; whether the job is still that version's compile comes from what is stored. If its last attempt throws, it records the version advisory, so a version never stays compiling. - `IRuleChecks`: - reads a version's compile and its proposals - proposes again, which marks the undecided proposals replaced - confirms a proposal, as proposed or edited, under its proposed name or another - rejects a proposal - for the job only: reads the version it compiles, and records what it found - **Infrastructure**: - Saving a rule version (create or revise) schedules its compile in the same unit of work: a job when the `rules` profile is configured, and otherwise an advisory compile at once. - Tables `RuleCompilations` and `RuleCheckProposals`, with migrations for SQLite and PostgreSQL; a proposal's definition is `jsonb` on PostgreSQL. - Confirming creates the check and activates it once its examples get their verdicts. The check, its `check.created` and `check.activated` events, the proposal's decision and `rule.check_confirmed` commit together. The examples run before the unit of work, so no model call happens inside it. - `CheckDirectory.ListActiveAsync` leaves out a check confirmed from a rule while the rule is not active at the version the check was proposed for. So the check run, `check_draft` and the inline-or-job decision all follow the rule's state. - Docs: CONTRIBUTING.md on the compile, and ADR-0012 now lists proposing checks among the check engine's actions. ## Why RUL-04 turns a rule's prose into checks the author can trust: the system proposes, a person decides (D17). The next slice labels rules enforced or advisory and gives agents the rule book with its compiled checks, which needs to know which checks serve which rule and version. Choices worth a look: - **Compiling is a job, as the check engine.** A model call can take seconds, so saving a rule never waits for one. The job runs as the check engine, since proposing checks is the engine's work, and is charged to the rule's author. A person's calls are uncapped, as for the check run. No new engine was added. - **A check serves the rule version it was confirmed for.** It guards content only while that version is active. A draft rule's checks wait for its set to publish it, a retirement stops them, a revert brings them back, and publishing a revision hands over to the checks confirmed for it. The cost is that a revision's checks are confirmed again, even for a small edit; that keeps "what runs" equal to "what the author confirmed for this text". - **Confirming also activates.** Activation is what runs a check's examples, and a person confirms (D17), so one step does both. The examples must get their verdicts, or nothing is stored. - **Proposal numbers are never reused.** Proposing again marks undecided proposals `replaced` rather than deleting them, so a number the author read always names the same proposal. It first moves the compile to its new job, which waits for a compile that is recording at the same time, so that compile's proposals are replaced too. - **Check names come from the rule**, as `<rule slug>-<number>`, not from the model. The author can give another name when confirming. - **Not in this PR**: enforced and advisory labels, the rule book tool and resource and rule hints (RUL-05, RUL-06, MCP-02, MCP-03), flagging instruction-like rule text (RUL-07), and proposing a rule from the desk (RUL-08). ## Tests Unit: - A voice and style rule compiles to a terms check, with the rule's severity and scope and its examples. The request goes to the `rules` profile on the author's behalf, with the rule's text and scope only in the data sections. - No model is configured: the rule is advisory and no call is made. - A failed call leaves the rule advisory, with the reason. - An answer that cannot be read leaves the rule advisory. An answer in a code fence is still read. - A proposal of an unknown or unoffered type, without a failing example, or with settings that are not an object is left out. - A rule no check can enforce is advisory. - At most three checks are proposed. - The rule compile runs as the check engine, and `rules.compile` passes the job handler conformance suite. - A compile payload whose version is not a whole number fails as `payload_invalid`. - A compile that throws is retried, and its last attempt leaves the version advisory with a message. Integration, each on SQLite and PostgreSQL: - Without a model, a saved rule is advisory with the message, and no compile is queued. - The compile job, run as the check engine for the author, proposes the two checks. The model call's system message does not hold the rule's text, which sits inside the user message's data delimiters. `rule.compiled` names the model and the prompt hash. - A confirmed check is created and active, with its events. It does not run while the rule is a draft. Once the set publishes the rule, "best in class" fails it; once the rule is retired, it does not run. - A revision's confirmed check takes over from the active version's when the revision is published. - An edited confirmation creates the edited check under another name. A confirmation whose examples fail, or whose name is taken, stores nothing. - A confirmed or rejected proposal cannot be decided again, and an unknown number is refused. - Proposing again marks the undecided proposals replaced and numbers new ones after them. The compile it replaced records nothing. - An unreadable answer leaves the version advisory. - An agent and an engine are refused confirming, rejecting and proposing again. A person and an agent are refused reading or recording a compile as the job does. The full suite ran locally against a `postgres:17` container as well as SQLite, and markdownlint passes. No new package. Tests use a fake model provider: no real model calls, no keys, no network. ## Checklist - [x] Linked to an issue a maintainer has confirmed - [x] Targets `v2` - [x] Tests or docs updated where behaviour changed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2634
What
The first slice of F-RUL (#2419), RUL-01 to RUL-03: the rule book, with rules stored as versioned content and published through change sets. It builds on the change set writes of #2520 (SCH-05).
FluentCMS.Core.Rules):RuleDefinitionholds four things, stored as one JSON document (ADR-0005), withValidatesaying what to change:Rule: a slug, a latest version and an active version. Its state is draft, active or retired, and only publishing moves it.IRuleDirectory:ChangeSetTarget: create, revise (from the latest, the active, or the version the set already proposes) and retireItemDiff.ForRule: the kind, text, severity and each scope list, before and after.RulesandRuleVersionstables, with migrations for SQLite and PostgreSQL; the definition isjsonbon PostgreSQL.RuleDirectorywrites asContentTypeDirectorydoes: every expected failure is found before the unit of work, and the version, itsrule.createdorrule.revisedevent and the change set operation commit together.ChangeSetDirectoryhandles rule items:rule.publishedorrule.unpublishedis loggedWhy
RUL-01 makes rules content: written, versioned, reviewed and published through change sets like everything else. The next slices propose checks from a rule and read the rule book for agents, and both need a rule to have versions and an active one.
Choices worth a look:
CheckSeverity, since the checks proposed from a rule take its severity.Tests
Unit:
Integration, each on SQLite and PostgreSQL:
rule.published.The full suite ran locally against a
postgres:17container as well as SQLite, and markdownlint passes. No new package. This PR adds noIContentTypeDirectory.CreateAsyncorReviseAsynccall.Checklist
v2