Skip to content

the judge is a bracketed token, and the brackets are part of the match - #446

Merged
Sunrisepeak merged 1 commit into
mainfrom
the-judge-is-a-bracketed-token-not-a-word
Sep 20, 2026
Merged

Sunrisepeak merged 1 commit into
mainfrom
the-judge-is-a-bracketed-token-not-a-word

Conversation

@Sunrisepeak

Copy link
Copy Markdown
Member

What this changes

compat.py classified a member as correctly refused whenever its output matched the bare word interface-not-provided. The comment above that line claimed the judge was "the engine's own refusal code and NOT a string in the diagnostic" — it was a string in the diagnostic.

The distinction the comment was reaching for is real. mcpp prints [interface-not-provided] in brackets, the way it prints E0006, and the token is an entry in docs/50-machine-output.md's table: a machine interface this index may read, rather than a sentence that may be rewritten. The brackets are what make it one, so they are now part of the match.

Why it matters

Read as a bare word, interface-not-provided is a hyphenated phrase an ordinary compile error can contain — an upstream message quoting a manifest key, or a member's own diagnostic naming the condition. A member that merely failed would be published as correctly refused, moving the compatibility figure in the direction that flatters the engine.

The rule now has a criterion

classify_failure is split out of measure so it can be exercised without a toolchain, a network or a member. compat.py selftest states five cases:

ok    a bracketed token is a refusal: refused
ok    the same token as prose is a failure: fails
ok    an ordinary compile error is a failure: fails
ok    a member that ran and failed its tests still built: builds
ok    the same output without a runner is not evidence it built: fails

Restoring the previous matcher fails the second and only the second:

FAIL  the same token as prose is a failure: refused (expected fails)
4 passed, 1 failed

The step runs in lint, not in openkal-compat.yml: its cases are about the rule rather than any member, and the measurement it guards takes four hours to reach.

`compat.py` recorded a member as correctly "refused" when its output matched
the bare word `interface-not-provided`, under a comment claiming the judge was
"the engine's own refusal code and NOT a string in the diagnostic". It was a
string in the diagnostic. What the comment was reaching for is true but was
not what the code did: mcpp prints `[interface-not-provided]` in brackets, the
way it prints `E0006`, and the token is an entry in docs/50's table -- a
machine interface this index may read rather than a sentence that may be
rewritten. The brackets are what make it one.

Read as a bare word it is a hyphenated phrase an ordinary compile error can
contain: an upstream message quoting a manifest key, or a member's own
diagnostic naming the condition. A member that merely failed would then be
published as correctly refused, which moves the figure in the direction that
flatters the engine.

`classify_failure` is split out of `measure` so the rule has a criterion that
runs without a toolchain, a network or a member, and `compat.py selftest`
states five: a bracketed token is a refusal, the same token as prose is a
failure, an ordinary compile error is a failure, a member that ran and failed
its tests still built, and the same output without a runner is not evidence it
built. Restoring the previous matcher fails the second and only the second.

The step runs in `lint` rather than in openkal-compat.yml: its cases are about
the rule, not about any member, and the measurement it guards takes four hours
to reach.
@Sunrisepeak
Sunrisepeak merged commit 090203d into main Sep 20, 2026
10 checks passed
@Sunrisepeak
Sunrisepeak deleted the the-judge-is-a-bracketed-token-not-a-word branch September 20, 2026 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant