diff --git a/artifacts/findings.yaml b/artifacts/findings.yaml index 899e742..0d1211e 100644 --- a/artifacts/findings.yaml +++ b/artifacts/findings.yaml @@ -6563,3 +6563,69 @@ artifacts: detected-by: clean-room verification of AFD-113 and AFD-114 with a fresh-context subagent, 2026-09-08 severity: major triage-status: confirmed + + - id: AFD-116 + type: ai-found-defect + title: "jess's cross-runtime kiln check was STRUCTURALLY INCAPABLE of passing — it invoked a function the fused core does not export, on a different artifact from the wasmtime half, and swallowed the error as 'inconclusive'; kiln 0.5.0 makes a real one possible" + status: open + description: |- + 2026-09-09. Per-piece release-watch of kiln v0.5.0 ("Run what we ship"), whose headline + SR-58 is `--invoke` on meld-fused CORE modules — exactly what jess ships. Testing it against + the real fused cascade found a defect in JESS, not in kiln. + + *** THE CHECK COULD NEVER HAVE PASSED *** + scripts/jess-build.sh ran: + kout="$("$KILND" "$FUSED" --function run-stabilization 2>/dev/null || true)" + and printed "kiln check inconclusive (non-gating)" whenever it produced nothing. It + produced nothing EVERY time, for three compounding reasons: + 1. The fused core exports no `run-stabilization`. meld's fusion renames the exports to + `pulseengine:falcon-cascade/@0.7.0#`. kiln says so plainly — + "[Runtime][E07DA] Function not found" — and then LISTS the five that exist. The + information needed to notice was in the output that `2>/dev/null` discarded. + 2. The wasmtime half of the "comparison" (line 73) invokes `run-stabilization()` on $W, + the PRE-FUSION component — a different artifact. The two halves were never looking at + the same thing, so even a working invocation would not have been a differential. + 3. `2>/dev/null || true` made "kiln disagreed" and "kiln could not run" the same reading. + That is the confusion varve#130 recorded when a publish-check reported exit 127 — + command not found — as "refused this push". + The JUnit evidence emitted a `kiln-xruntime` testcase carrying KILN_OK, which was + necessarily 0 on every run this repo has ever done. + + *** WHAT KILN 0.5.0 ACTUALLY GIVES US, MEASURED *** + CLI conventions now met (they were not in the layer's build): `kilnd --version` prints + `kilnd 0.5.0` and an unknown flag exits 2. `--invoke` + `--arg` work on jess's real + meld-fused core: invoking `pulseengine:falcon-cascade/mixer@0.7.0#mix` with four f32s + returns `i32 9488`, and wasmtime returns 9488 for the same module, export and arguments. + All FIVE stage exports execute under kiln when each is driven with its own arity. + + *** AND WHAT IT DOES NOT GIVE US — CHECKED BEFORE BUILDING ON IT *** + Every cascade export returns an i32 that is a POINTER into the return area, and that + pointer is CONSTANT: 9488 for three different argument sets (1.0/0/0/0.5, 0/1.0/0/0.9, + 0.25/0.25/0.25/0.1). Two engines agreeing on a fixed address is NOT evidence about the + arithmetic — a check built on it would agree even if one engine computed garbage. So the + replacement is deliberately labelled an EXECUTION check, not a value differential. Making + it a value differential needs kiln to dereference the return area (read N bytes at the + returned pointer); filed upstream. + This is the vacuity rule applied BEFORE writing the checker rather than after: the reason + the new check is modest is that the strong version was measured to be unavailable. + + *** THE REPLACEMENT *** + tools/xruntime/kiln-check.sh drives BOTH engines over the SAME artifact, export and + arguments; requires a value from each (an empty reading is "could not run", not + agreement); refuses a pre-0.5.0 kilnd by name rather than reporting "inconclusive"; and + asserts all 5/5 stage exports execute. jess-build.sh passes MOD="$FUSED" KILND="$KILND" + explicitly — letting the checker fall back to its own defaults would have checked a + different file than the one just fused, reintroducing defect (2) one layer down. + Still non-gating for the build unless KILN_STRICT=1; promoting it is a separate decision. + Negative controls, both observed firing: a pre-0.5.0 kilnd is REFUSED with its banner + quoted, and a module without the export FAILS instead of passing. + Two harness bugs of jess's own, caught by the controls: the comparison export name was + packed as ":4" and split on the FIRST colon — but the name itself contains colons, + so it truncated to "pulseengine"; and driving all five exports with six zeros made four + fail on arity and reported "1/5", a number that measures the harness rather than the + artifact. + tags: [kiln, release-watch, vacuous-gate, cross-runtime, wasmtime, varve-130, evidence] + fields: + detected-by: per-piece release-watch of kiln v0.5.0 against the real fused cascade, 2026-09-09 + severity: major + triage-status: confirmed diff --git a/scripts/jess-build.sh b/scripts/jess-build.sh index 2f01ad4..d458f0e 100755 --- a/scripts/jess-build.sh +++ b/scripts/jess-build.sh @@ -99,17 +99,27 @@ echo " ELF: $(file "$ELF" | cut -d, -f1-2)" # --- cross-runtime check: kiln on fused core vs wasmtime ----------------- note "cross-runtime check (kiln on fused core)" +# The previous version invoked `--function run-stabilization` on $FUSED and reported +# "inconclusive (non-gating)" on any failure. It failed EVERY time and could not have done +# otherwise: the fused core exports no such function (meld renames them to +# pulseengine:falcon-cascade/@0.7.0#, and kiln says so and lists them), while the +# wasmtime side above invokes that name on $W — a DIFFERENT artifact. So the two halves never +# compared the same thing, `2>/dev/null || true` made "disagreed" and "could not run" +# identical, and the JUnit `kiln-xruntime` case has always recorded 0. See AFD-116. KILN_OK=0 -kout="$("$KILND" "$FUSED" --function run-stabilization 2>/dev/null || true)" -kbits="$(printf '%s' "$kout" | sed -nE 's/.*FloatBits32\(([0-9]+)\).*/\1/p' | head -1)" -if [ -n "$kbits" ]; then - kval="$(python3 -c "import struct;print(struct.unpack('/dev/null || true)" - if [ -n "$kval" ]; then - echo " kiln run-stabilization = $kval rad (wasmtime $STAB)" - KILN_OK="$(python3 -c "print(1 if abs(float('$kval')-float('$STAB'))<1e-4 else 0)" 2>/dev/null || echo 0)" - fi +# Pass the artifact and binary THIS script built/uses. Letting the checker fall back to its +# own defaults would check a different file than the one just fused — which is the exact +# defect being fixed here, reintroduced one layer down. +if kout="$(MOD="$FUSED" KILND="$KILND" "$ROOT/tools/xruntime/kiln-check.sh" 2>&1)"; then + KILN_OK=1; printf '%s\n' "$kout" | sed 's/^/ /' +else + printf '%s\n' "$kout" | sed 's/^/ /' + # NOT "inconclusive": the reason is printed above and distinguishes "could not run" from + # "disagreed". Still non-gating for the overall build unless KILN_STRICT=1 — promoting it + # to a gate is a separate, reviewable decision. + [ "${KILN_STRICT:-0}" = 1 ] && fail "cross-runtime kiln check failed (KILN_STRICT=1)" + echo " kiln cross-runtime check FAILED (non-gating; set KILN_STRICT=1 to gate)" fi -[ "$KILN_OK" = 1 ] && echo " kiln matches wasmtime" || echo " kiln check inconclusive (non-gating)" # --- emit JUnit evidence ------------------------------------------------- note "evidence" diff --git a/tools/xruntime/kiln-check.sh b/tools/xruntime/kiln-check.sh new file mode 100755 index 0000000..49125a2 --- /dev/null +++ b/tools/xruntime/kiln-check.sh @@ -0,0 +1,101 @@ +#!/usr/bin/env bash +# Cross-runtime check: does kiln execute the SHIPPED meld-fused core the same way wasmtime does? +# +# WHAT THIS REPLACES. scripts/jess-build.sh ran +# "$KILND" "$FUSED" --function run-stabilization 2>/dev/null || true +# and reported "kiln check inconclusive (non-gating)" whenever it failed. It failed EVERY time, +# and could not have done otherwise: +# * the fused core exports no `run-stabilization`. kiln says so plainly — +# "[Runtime][E07DA] Function not found" — and then LISTS the five exports it does have. +# meld's fusion renames them to `pulseengine:falcon-cascade/@0.7.0#`. +# * wasmtime's side of the "comparison" invoked that name on a DIFFERENT artifact (the +# pre-fusion component), so the two halves were never looking at the same thing. +# * `2>/dev/null || true` then made "kiln disagreed" and "kiln could not run" the same +# reading — the exact confusion varve#130 recorded as reporting exit 127 as a refusal. +# The JUnit evidence still carried a `kiln-xruntime` testcase whose value was always 0. +# +# WHAT THIS CLAIMS, AND WHAT IT DOES NOT. kiln 0.5.0 (SR-58) can invoke exports on a +# meld-fused CORE module, which is what jess ships. Both engines are driven over the SAME +# artifact, the SAME export and the SAME arguments, and must agree. +# +# It is NOT a value differential, and saying so matters: every cascade export returns an i32 +# that is a POINTER into the return area, and that pointer was MEASURED to be constant (9488) +# across three different argument sets. Two engines agreeing on a fixed address is not +# evidence about the arithmetic. What this does establish is that a second, independent engine +# LOADS AND EXECUTES the shipped fused core across all five stage exports without trapping, +# and returns what wasmtime returns. Modest, but true — and it is a real executed result where +# the previous check was a permanently-failing no-op. +# +# Making it a value differential needs kiln to dereference the return area (read N bytes at the +# returned pointer). Filed upstream; until then this check is deliberately labelled down. +set -uo pipefail +ROOT="$(cd "$(dirname "$0")/../.." && pwd -P)" +KILND="${KILND:-kilnd}" +MOD="${MOD:-$ROOT/.scratch/invoke/c.loom.wasm}" +fail() { printf 'FAIL: %s\n' "$*" >&2; exit 1; } + +# The export name itself contains colons, so it is NOT safe to pack a field after one. +# A first version wrote "…#mix:4" and split on the FIRST colon, yielding the export name +# "pulseengine" — which then failed as "could not run", correctly, rather than silently +# comparing nothing. +CMP_EXPORT="pulseengine:falcon-cascade/mixer@0.7.0#mix" +ARGS4="0.25 0.5 0.75 0.125" + +invoke_kiln() { # $1 export, rest args -> prints the i32 or nothing + local e="$1"; shift + local a=() x + for x in "$@"; do a+=(--arg "$x"); done + "$KILND" "$MOD" --invoke "$e" "${a[@]}" 2>&1 | sed -n 's/.*\[0\] i32 \([0-9-]*\).*/\1/p' | head -1 +} +invoke_wasmtime() { local e="$1"; shift; wasmtime run --invoke "$e" "$MOD" "$@" 2>/dev/null | tail -1; } + +[ -f "$MOD" ] || fail "fused core not found: $MOD" +command -v wasmtime >/dev/null || fail "wasmtime not on PATH" +command -v "$KILND" >/dev/null || [ -x "$KILND" ] \ + || fail "kilnd not found at '$KILND' — that is 'could not run the check', NOT 'the check failed'" + +# kilnd must be >= 0.5.0: earlier builds have no --invoke and cannot reach a fused core at all. +kv="$("$KILND" --version 2>&1 | head -1)" +case "$kv" in + "kilnd "*) : ;; + *) fail "kilnd does not report a version ('$kv'). Pre-0.5.0 builds print a banner instead and + have no --invoke, so this check cannot run against them — refusing rather than reporting + 'inconclusive', which is how the previous version hid a permanent failure." ;; +esac +echo " kiln: $kv" + +e="$CMP_EXPORT" +k="$(invoke_kiln "$e" $ARGS4)" +w="$(invoke_wasmtime "$e" $ARGS4)" +# BOTH must produce a value. An empty reading is "could not run"; treating it as agreement is +# precisely the defect being fixed here. +[ -n "$k" ] || fail "kiln produced no value for $e — could not run" +[ -n "$w" ] || fail "wasmtime produced no value for $e — could not run" +[ "$k" = "$w" ] || fail "CROSS-RUNTIME DISAGREEMENT on $e: kiln=$k wasmtime=$w" +echo " $e -> kiln=$k wasmtime=$w AGREE" + +# And every stage export must at least LOAD AND EXECUTE under kiln. This is the part that is +# genuinely about the shipped artifact rather than about one function. +# Each export is driven with ITS OWN arity. The cascade is not uniform — three distinct +# shapes, measured from the fused module's core signatures: +# ekf#estimate (param f32 x6) -> i32 position/attitude/rate #tick (param i32) -> i32 +# mixer#mix (param f32 x4) -> i32 +# A first version passed six zeros to all five, so four of them failed on arity and the count +# read 1/5 — a number that says nothing about the artifact. Passing the wrong arity everywhere +# and then reporting the survivors is the shape of a metric that measures the harness. +n=0; total=0; failed="" +for spec in "ekf@0.7.0#estimate:0 0 0 0 0 0" \ + "position@0.7.0#tick:0" \ + "attitude@0.7.0#tick:0" \ + "mixer@0.7.0#mix:0.25 0.5 0.75 0.125" \ + "rate@0.7.0#tick:0"; do + stage="${spec%%:*}"; a="${spec#*:}" + full="pulseengine:falcon-cascade/$stage" + total=$((total+1)) + if "$KILND" "$MOD" --invoke "$full" $(for x in $a; do printf -- '--arg %s ' "$x"; done) >/dev/null 2>&1 + then n=$((n+1)); else failed="$failed $stage"; fi +done +[ "$n" -eq "$total" ] || fail "kiln executed only $n/$total stage exports; failed:$failed" +echo " kiln executed $n/$total stage exports of the shipped fused core" +echo "NOTE: agreement is on a return-area POINTER, measured constant across inputs — this is an" +echo " EXECUTION check, not a value differential (see the header)."