From 14c2c3179a66c52a156f719e6fd9a296b9a7c4d2 Mon Sep 17 00:00:00 2001 From: Ralf Anton Beier Date: Wed, 9 Sep 2026 00:49:42 +0200 Subject: [PATCH] =?UTF-8?q?fix:=20the=20cross-runtime=20kiln=20check=20was?= =?UTF-8?q?=20structurally=20incapable=20of=20passing=20=E2=80=94=20wrong?= =?UTF-8?q?=20function,=20wrong=20artifact,=20error=20swallowed=20(AFD-116?= =?UTF-8?q?)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found by the per-piece release-watch of kiln v0.5.0, whose headline SR-58 is --invoke on meld-fused CORE modules — exactly what jess ships. Testing it against the real fused cascade found a defect in JESS, not in kiln. scripts/jess-build.sh ran: kout="$("$KILND" "$FUSED" --function run-stabilization 2>/dev/null || true)" and printed "kiln check inconclusive (non-gating)" whenever it produced nothing. It produced nothing every time, for three compounding reasons: 1. the fused core exports no `run-stabilization` — meld renames the exports to pulseengine:falcon-cascade/@0.7.0#. kiln says so plainly ("[Runtime][E07DA] Function not found") and then LISTS the five that exist. The information needed to notice was in the output 2>/dev/null discarded. 2. the wasmtime half invoked that name on $W, the PRE-FUSION component — a different artifact. The two halves were never comparing the same thing. 3. `2>/dev/null || true` made "kiln disagreed" and "kiln could not run" the same reading — the confusion varve#130 recorded when exit 127 was reported as a refusal. The JUnit evidence carried a kiln-xruntime testcase whose value was necessarily 0 on every run this repo has ever done. WHAT 0.5.0 GIVES US, MEASURED: kilnd --version now prints `kilnd 0.5.0` and an unknown flag exits 2 (neither held in the layer's build). --invoke works on the real fused core: mixer#mix with four f32s returns i32 9488, and wasmtime returns 9488 for the same module, export and args. All five stage exports execute when each is driven with its own arity. WHAT IT DOES NOT GIVE US — checked BEFORE building on it: every cascade export returns a POINTER, and that pointer is CONSTANT (9488) across three different argument sets. Two engines agreeing on a fixed address is not evidence about the arithmetic; a check built on it would agree even if one engine computed garbage. So the replacement is labelled an EXECUTION check, not a value differential. The strong version needs kiln to dereference the return area — filed upstream. The replacement drives both engines over the SAME artifact/export/args, requires a value from each, refuses a pre-0.5.0 kilnd by name instead of saying "inconclusive", and asserts 5/5 stage exports execute. jess-build.sh passes MOD="$FUSED" KILND="$KILND" explicitly — letting the checker use its own defaults would have reintroduced defect (2) one layer down. Non-gating unless KILN_STRICT=1. Both negative controls observed firing. Two harness bugs of my own caught by them: the export name was packed as ":4" and split on the first colon, but the name contains colons and truncated to "pulseengine"; and driving all five exports with six zeros made four fail on arity and reported "1/5" — a number measuring the harness, not the artifact. Co-Authored-By: Claude Opus 4.8 --- artifacts/findings.yaml | 66 +++++++++++++++++++++++ scripts/jess-build.sh | 28 ++++++---- tools/xruntime/kiln-check.sh | 101 +++++++++++++++++++++++++++++++++++ 3 files changed, 186 insertions(+), 9 deletions(-) create mode 100755 tools/xruntime/kiln-check.sh diff --git a/artifacts/findings.yaml b/artifacts/findings.yaml index 899e742..0d1211e 100644 --- a/artifacts/findings.yaml +++ b/artifacts/findings.yaml @@ -6563,3 +6563,69 @@ artifacts: detected-by: clean-room verification of AFD-113 and AFD-114 with a fresh-context subagent, 2026-09-08 severity: major triage-status: confirmed + + - id: AFD-116 + type: ai-found-defect + title: "jess's cross-runtime kiln check was STRUCTURALLY INCAPABLE of passing — it invoked a function the fused core does not export, on a different artifact from the wasmtime half, and swallowed the error as 'inconclusive'; kiln 0.5.0 makes a real one possible" + status: open + description: |- + 2026-09-09. Per-piece release-watch of kiln v0.5.0 ("Run what we ship"), whose headline + SR-58 is `--invoke` on meld-fused CORE modules — exactly what jess ships. Testing it against + the real fused cascade found a defect in JESS, not in kiln. + + *** THE CHECK COULD NEVER HAVE PASSED *** + scripts/jess-build.sh ran: + kout="$("$KILND" "$FUSED" --function run-stabilization 2>/dev/null || true)" + and printed "kiln check inconclusive (non-gating)" whenever it produced nothing. It + produced nothing EVERY time, for three compounding reasons: + 1. The fused core exports no `run-stabilization`. meld's fusion renames the exports to + `pulseengine:falcon-cascade/@0.7.0#`. kiln says so plainly — + "[Runtime][E07DA] Function not found" — and then LISTS the five that exist. The + information needed to notice was in the output that `2>/dev/null` discarded. + 2. The wasmtime half of the "comparison" (line 73) invokes `run-stabilization()` on $W, + the PRE-FUSION component — a different artifact. The two halves were never looking at + the same thing, so even a working invocation would not have been a differential. + 3. `2>/dev/null || true` made "kiln disagreed" and "kiln could not run" the same reading. + That is the confusion varve#130 recorded when a publish-check reported exit 127 — + command not found — as "refused this push". + The JUnit evidence emitted a `kiln-xruntime` testcase carrying KILN_OK, which was + necessarily 0 on every run this repo has ever done. + + *** WHAT KILN 0.5.0 ACTUALLY GIVES US, MEASURED *** + CLI conventions now met (they were not in the layer's build): `kilnd --version` prints + `kilnd 0.5.0` and an unknown flag exits 2. `--invoke` + `--arg` work on jess's real + meld-fused core: invoking `pulseengine:falcon-cascade/mixer@0.7.0#mix` with four f32s + returns `i32 9488`, and wasmtime returns 9488 for the same module, export and arguments. + All FIVE stage exports execute under kiln when each is driven with its own arity. + + *** AND WHAT IT DOES NOT GIVE US — CHECKED BEFORE BUILDING ON IT *** + Every cascade export returns an i32 that is a POINTER into the return area, and that + pointer is CONSTANT: 9488 for three different argument sets (1.0/0/0/0.5, 0/1.0/0/0.9, + 0.25/0.25/0.25/0.1). Two engines agreeing on a fixed address is NOT evidence about the + arithmetic — a check built on it would agree even if one engine computed garbage. So the + replacement is deliberately labelled an EXECUTION check, not a value differential. Making + it a value differential needs kiln to dereference the return area (read N bytes at the + returned pointer); filed upstream. + This is the vacuity rule applied BEFORE writing the checker rather than after: the reason + the new check is modest is that the strong version was measured to be unavailable. + + *** THE REPLACEMENT *** + tools/xruntime/kiln-check.sh drives BOTH engines over the SAME artifact, export and + arguments; requires a value from each (an empty reading is "could not run", not + agreement); refuses a pre-0.5.0 kilnd by name rather than reporting "inconclusive"; and + asserts all 5/5 stage exports execute. jess-build.sh passes MOD="$FUSED" KILND="$KILND" + explicitly — letting the checker fall back to its own defaults would have checked a + different file than the one just fused, reintroducing defect (2) one layer down. + Still non-gating for the build unless KILN_STRICT=1; promoting it is a separate decision. + Negative controls, both observed firing: a pre-0.5.0 kilnd is REFUSED with its banner + quoted, and a module without the export FAILS instead of passing. + Two harness bugs of jess's own, caught by the controls: the comparison export name was + packed as ":4" and split on the FIRST colon — but the name itself contains colons, + so it truncated to "pulseengine"; and driving all five exports with six zeros made four + fail on arity and reported "1/5", a number that measures the harness rather than the + artifact. + tags: [kiln, release-watch, vacuous-gate, cross-runtime, wasmtime, varve-130, evidence] + fields: + detected-by: per-piece release-watch of kiln v0.5.0 against the real fused cascade, 2026-09-09 + severity: major + triage-status: confirmed diff --git a/scripts/jess-build.sh b/scripts/jess-build.sh index 2f01ad4..d458f0e 100755 --- a/scripts/jess-build.sh +++ b/scripts/jess-build.sh @@ -99,17 +99,27 @@ echo " ELF: $(file "$ELF" | cut -d, -f1-2)" # --- cross-runtime check: kiln on fused core vs wasmtime ----------------- note "cross-runtime check (kiln on fused core)" +# The previous version invoked `--function run-stabilization` on $FUSED and reported +# "inconclusive (non-gating)" on any failure. It failed EVERY time and could not have done +# otherwise: the fused core exports no such function (meld renames them to +# pulseengine:falcon-cascade/@0.7.0#, and kiln says so and lists them), while the +# wasmtime side above invokes that name on $W — a DIFFERENT artifact. So the two halves never +# compared the same thing, `2>/dev/null || true` made "disagreed" and "could not run" +# identical, and the JUnit `kiln-xruntime` case has always recorded 0. See AFD-116. KILN_OK=0 -kout="$("$KILND" "$FUSED" --function run-stabilization 2>/dev/null || true)" -kbits="$(printf '%s' "$kout" | sed -nE 's/.*FloatBits32\(([0-9]+)\).*/\1/p' | head -1)" -if [ -n "$kbits" ]; then - kval="$(python3 -c "import struct;print(struct.unpack('/dev/null || true)" - if [ -n "$kval" ]; then - echo " kiln run-stabilization = $kval rad (wasmtime $STAB)" - KILN_OK="$(python3 -c "print(1 if abs(float('$kval')-float('$STAB'))<1e-4 else 0)" 2>/dev/null || echo 0)" - fi +# Pass the artifact and binary THIS script built/uses. Letting the checker fall back to its +# own defaults would check a different file than the one just fused — which is the exact +# defect being fixed here, reintroduced one layer down. +if kout="$(MOD="$FUSED" KILND="$KILND" "$ROOT/tools/xruntime/kiln-check.sh" 2>&1)"; then + KILN_OK=1; printf '%s\n' "$kout" | sed 's/^/ /' +else + printf '%s\n' "$kout" | sed 's/^/ /' + # NOT "inconclusive": the reason is printed above and distinguishes "could not run" from + # "disagreed". Still non-gating for the overall build unless KILN_STRICT=1 — promoting it + # to a gate is a separate, reviewable decision. + [ "${KILN_STRICT:-0}" = 1 ] && fail "cross-runtime kiln check failed (KILN_STRICT=1)" + echo " kiln cross-runtime check FAILED (non-gating; set KILN_STRICT=1 to gate)" fi -[ "$KILN_OK" = 1 ] && echo " kiln matches wasmtime" || echo " kiln check inconclusive (non-gating)" # --- emit JUnit evidence ------------------------------------------------- note "evidence" diff --git a/tools/xruntime/kiln-check.sh b/tools/xruntime/kiln-check.sh new file mode 100755 index 0000000..49125a2 --- /dev/null +++ b/tools/xruntime/kiln-check.sh @@ -0,0 +1,101 @@ +#!/usr/bin/env bash +# Cross-runtime check: does kiln execute the SHIPPED meld-fused core the same way wasmtime does? +# +# WHAT THIS REPLACES. scripts/jess-build.sh ran +# "$KILND" "$FUSED" --function run-stabilization 2>/dev/null || true +# and reported "kiln check inconclusive (non-gating)" whenever it failed. It failed EVERY time, +# and could not have done otherwise: +# * the fused core exports no `run-stabilization`. kiln says so plainly — +# "[Runtime][E07DA] Function not found" — and then LISTS the five exports it does have. +# meld's fusion renames them to `pulseengine:falcon-cascade/@0.7.0#`. +# * wasmtime's side of the "comparison" invoked that name on a DIFFERENT artifact (the +# pre-fusion component), so the two halves were never looking at the same thing. +# * `2>/dev/null || true` then made "kiln disagreed" and "kiln could not run" the same +# reading — the exact confusion varve#130 recorded as reporting exit 127 as a refusal. +# The JUnit evidence still carried a `kiln-xruntime` testcase whose value was always 0. +# +# WHAT THIS CLAIMS, AND WHAT IT DOES NOT. kiln 0.5.0 (SR-58) can invoke exports on a +# meld-fused CORE module, which is what jess ships. Both engines are driven over the SAME +# artifact, the SAME export and the SAME arguments, and must agree. +# +# It is NOT a value differential, and saying so matters: every cascade export returns an i32 +# that is a POINTER into the return area, and that pointer was MEASURED to be constant (9488) +# across three different argument sets. Two engines agreeing on a fixed address is not +# evidence about the arithmetic. What this does establish is that a second, independent engine +# LOADS AND EXECUTES the shipped fused core across all five stage exports without trapping, +# and returns what wasmtime returns. Modest, but true — and it is a real executed result where +# the previous check was a permanently-failing no-op. +# +# Making it a value differential needs kiln to dereference the return area (read N bytes at the +# returned pointer). Filed upstream; until then this check is deliberately labelled down. +set -uo pipefail +ROOT="$(cd "$(dirname "$0")/../.." && pwd -P)" +KILND="${KILND:-kilnd}" +MOD="${MOD:-$ROOT/.scratch/invoke/c.loom.wasm}" +fail() { printf 'FAIL: %s\n' "$*" >&2; exit 1; } + +# The export name itself contains colons, so it is NOT safe to pack a field after one. +# A first version wrote "…#mix:4" and split on the FIRST colon, yielding the export name +# "pulseengine" — which then failed as "could not run", correctly, rather than silently +# comparing nothing. +CMP_EXPORT="pulseengine:falcon-cascade/mixer@0.7.0#mix" +ARGS4="0.25 0.5 0.75 0.125" + +invoke_kiln() { # $1 export, rest args -> prints the i32 or nothing + local e="$1"; shift + local a=() x + for x in "$@"; do a+=(--arg "$x"); done + "$KILND" "$MOD" --invoke "$e" "${a[@]}" 2>&1 | sed -n 's/.*\[0\] i32 \([0-9-]*\).*/\1/p' | head -1 +} +invoke_wasmtime() { local e="$1"; shift; wasmtime run --invoke "$e" "$MOD" "$@" 2>/dev/null | tail -1; } + +[ -f "$MOD" ] || fail "fused core not found: $MOD" +command -v wasmtime >/dev/null || fail "wasmtime not on PATH" +command -v "$KILND" >/dev/null || [ -x "$KILND" ] \ + || fail "kilnd not found at '$KILND' — that is 'could not run the check', NOT 'the check failed'" + +# kilnd must be >= 0.5.0: earlier builds have no --invoke and cannot reach a fused core at all. +kv="$("$KILND" --version 2>&1 | head -1)" +case "$kv" in + "kilnd "*) : ;; + *) fail "kilnd does not report a version ('$kv'). Pre-0.5.0 builds print a banner instead and + have no --invoke, so this check cannot run against them — refusing rather than reporting + 'inconclusive', which is how the previous version hid a permanent failure." ;; +esac +echo " kiln: $kv" + +e="$CMP_EXPORT" +k="$(invoke_kiln "$e" $ARGS4)" +w="$(invoke_wasmtime "$e" $ARGS4)" +# BOTH must produce a value. An empty reading is "could not run"; treating it as agreement is +# precisely the defect being fixed here. +[ -n "$k" ] || fail "kiln produced no value for $e — could not run" +[ -n "$w" ] || fail "wasmtime produced no value for $e — could not run" +[ "$k" = "$w" ] || fail "CROSS-RUNTIME DISAGREEMENT on $e: kiln=$k wasmtime=$w" +echo " $e -> kiln=$k wasmtime=$w AGREE" + +# And every stage export must at least LOAD AND EXECUTE under kiln. This is the part that is +# genuinely about the shipped artifact rather than about one function. +# Each export is driven with ITS OWN arity. The cascade is not uniform — three distinct +# shapes, measured from the fused module's core signatures: +# ekf#estimate (param f32 x6) -> i32 position/attitude/rate #tick (param i32) -> i32 +# mixer#mix (param f32 x4) -> i32 +# A first version passed six zeros to all five, so four of them failed on arity and the count +# read 1/5 — a number that says nothing about the artifact. Passing the wrong arity everywhere +# and then reporting the survivors is the shape of a metric that measures the harness. +n=0; total=0; failed="" +for spec in "ekf@0.7.0#estimate:0 0 0 0 0 0" \ + "position@0.7.0#tick:0" \ + "attitude@0.7.0#tick:0" \ + "mixer@0.7.0#mix:0.25 0.5 0.75 0.125" \ + "rate@0.7.0#tick:0"; do + stage="${spec%%:*}"; a="${spec#*:}" + full="pulseengine:falcon-cascade/$stage" + total=$((total+1)) + if "$KILND" "$MOD" --invoke "$full" $(for x in $a; do printf -- '--arg %s ' "$x"; done) >/dev/null 2>&1 + then n=$((n+1)); else failed="$failed $stage"; fi +done +[ "$n" -eq "$total" ] || fail "kiln executed only $n/$total stage exports; failed:$failed" +echo " kiln executed $n/$total stage exports of the shipped fused core" +echo "NOTE: agreement is on a return-area POINTER, measured constant across inputs — this is an" +echo " EXECUTION check, not a value differential (see the header)."