Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions artifacts/findings.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6563,3 +6563,69 @@ artifacts:
detected-by: clean-room verification of AFD-113 and AFD-114 with a fresh-context subagent, 2026-09-08
severity: major
triage-status: confirmed

- id: AFD-116
type: ai-found-defect
title: "jess's cross-runtime kiln check was STRUCTURALLY INCAPABLE of passing — it invoked a function the fused core does not export, on a different artifact from the wasmtime half, and swallowed the error as 'inconclusive'; kiln 0.5.0 makes a real one possible"
status: open
description: |-
2026-09-09. Per-piece release-watch of kiln v0.5.0 ("Run what we ship"), whose headline
SR-58 is `--invoke` on meld-fused CORE modules — exactly what jess ships. Testing it against
the real fused cascade found a defect in JESS, not in kiln.

*** THE CHECK COULD NEVER HAVE PASSED ***
scripts/jess-build.sh ran:
kout="$("$KILND" "$FUSED" --function run-stabilization 2>/dev/null || true)"
and printed "kiln check inconclusive (non-gating)" whenever it produced nothing. It
produced nothing EVERY time, for three compounding reasons:
1. The fused core exports no `run-stabilization`. meld's fusion renames the exports to
`pulseengine:falcon-cascade/<stage>@0.7.0#<fn>`. kiln says so plainly —
"[Runtime][E07DA] Function not found" — and then LISTS the five that exist. The
information needed to notice was in the output that `2>/dev/null` discarded.
2. The wasmtime half of the "comparison" (line 73) invokes `run-stabilization()` on $W,
the PRE-FUSION component — a different artifact. The two halves were never looking at
the same thing, so even a working invocation would not have been a differential.
3. `2>/dev/null || true` made "kiln disagreed" and "kiln could not run" the same reading.
That is the confusion varve#130 recorded when a publish-check reported exit 127 —
command not found — as "refused this push".
The JUnit evidence emitted a `kiln-xruntime` testcase carrying KILN_OK, which was
necessarily 0 on every run this repo has ever done.

*** WHAT KILN 0.5.0 ACTUALLY GIVES US, MEASURED ***
CLI conventions now met (they were not in the layer's build): `kilnd --version` prints
`kilnd 0.5.0` and an unknown flag exits 2. `--invoke` + `--arg` work on jess's real
meld-fused core: invoking `pulseengine:falcon-cascade/mixer@0.7.0#mix` with four f32s
returns `i32 9488`, and wasmtime returns 9488 for the same module, export and arguments.
All FIVE stage exports execute under kiln when each is driven with its own arity.

*** AND WHAT IT DOES NOT GIVE US — CHECKED BEFORE BUILDING ON IT ***
Every cascade export returns an i32 that is a POINTER into the return area, and that
pointer is CONSTANT: 9488 for three different argument sets (1.0/0/0/0.5, 0/1.0/0/0.9,
0.25/0.25/0.25/0.1). Two engines agreeing on a fixed address is NOT evidence about the
arithmetic — a check built on it would agree even if one engine computed garbage. So the
replacement is deliberately labelled an EXECUTION check, not a value differential. Making
it a value differential needs kiln to dereference the return area (read N bytes at the
returned pointer); filed upstream.
This is the vacuity rule applied BEFORE writing the checker rather than after: the reason
the new check is modest is that the strong version was measured to be unavailable.

*** THE REPLACEMENT ***
tools/xruntime/kiln-check.sh drives BOTH engines over the SAME artifact, export and
arguments; requires a value from each (an empty reading is "could not run", not
agreement); refuses a pre-0.5.0 kilnd by name rather than reporting "inconclusive"; and
asserts all 5/5 stage exports execute. jess-build.sh passes MOD="$FUSED" KILND="$KILND"
explicitly — letting the checker fall back to its own defaults would have checked a
different file than the one just fused, reintroducing defect (2) one layer down.
Still non-gating for the build unless KILN_STRICT=1; promoting it is a separate decision.
Negative controls, both observed firing: a pre-0.5.0 kilnd is REFUSED with its banner
quoted, and a module without the export FAILS instead of passing.
Two harness bugs of jess's own, caught by the controls: the comparison export name was
packed as "<name>:4" and split on the FIRST colon — but the name itself contains colons,
so it truncated to "pulseengine"; and driving all five exports with six zeros made four
fail on arity and reported "1/5", a number that measures the harness rather than the
artifact.
tags: [kiln, release-watch, vacuous-gate, cross-runtime, wasmtime, varve-130, evidence]
fields:
detected-by: per-piece release-watch of kiln v0.5.0 against the real fused cascade, 2026-09-09
severity: major
triage-status: confirmed
28 changes: 19 additions & 9 deletions scripts/jess-build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -99,17 +99,27 @@ echo " ELF: $(file "$ELF" | cut -d, -f1-2)"

# --- cross-runtime check: kiln on fused core vs wasmtime -----------------
note "cross-runtime check (kiln on fused core)"
# The previous version invoked `--function run-stabilization` on $FUSED and reported
# "inconclusive (non-gating)" on any failure. It failed EVERY time and could not have done
# otherwise: the fused core exports no such function (meld renames them to
# pulseengine:falcon-cascade/<stage>@0.7.0#<fn>, and kiln says so and lists them), while the
# wasmtime side above invokes that name on $W — a DIFFERENT artifact. So the two halves never
# compared the same thing, `2>/dev/null || true` made "disagreed" and "could not run"
# identical, and the JUnit `kiln-xruntime` case has always recorded 0. See AFD-116.
KILN_OK=0
kout="$("$KILND" "$FUSED" --function run-stabilization 2>/dev/null || true)"
kbits="$(printf '%s' "$kout" | sed -nE 's/.*FloatBits32\(([0-9]+)\).*/\1/p' | head -1)"
if [ -n "$kbits" ]; then
kval="$(python3 -c "import struct;print(struct.unpack('<f',struct.pack('<I',$kbits))[0])" 2>/dev/null || true)"
if [ -n "$kval" ]; then
echo " kiln run-stabilization = $kval rad (wasmtime $STAB)"
KILN_OK="$(python3 -c "print(1 if abs(float('$kval')-float('$STAB'))<1e-4 else 0)" 2>/dev/null || echo 0)"
fi
# Pass the artifact and binary THIS script built/uses. Letting the checker fall back to its
# own defaults would check a different file than the one just fused — which is the exact
# defect being fixed here, reintroduced one layer down.
if kout="$(MOD="$FUSED" KILND="$KILND" "$ROOT/tools/xruntime/kiln-check.sh" 2>&1)"; then
KILN_OK=1; printf '%s\n' "$kout" | sed 's/^/ /'
else
printf '%s\n' "$kout" | sed 's/^/ /'
# NOT "inconclusive": the reason is printed above and distinguishes "could not run" from
# "disagreed". Still non-gating for the overall build unless KILN_STRICT=1 — promoting it
# to a gate is a separate, reviewable decision.
[ "${KILN_STRICT:-0}" = 1 ] && fail "cross-runtime kiln check failed (KILN_STRICT=1)"
echo " kiln cross-runtime check FAILED (non-gating; set KILN_STRICT=1 to gate)"
fi
[ "$KILN_OK" = 1 ] && echo " kiln matches wasmtime" || echo " kiln check inconclusive (non-gating)"

# --- emit JUnit evidence -------------------------------------------------
note "evidence"
Expand Down
101 changes: 101 additions & 0 deletions tools/xruntime/kiln-check.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
#!/usr/bin/env bash
# Cross-runtime check: does kiln execute the SHIPPED meld-fused core the same way wasmtime does?
#
# WHAT THIS REPLACES. scripts/jess-build.sh ran
# "$KILND" "$FUSED" --function run-stabilization 2>/dev/null || true
# and reported "kiln check inconclusive (non-gating)" whenever it failed. It failed EVERY time,
# and could not have done otherwise:
# * the fused core exports no `run-stabilization`. kiln says so plainly —
# "[Runtime][E07DA] Function not found" — and then LISTS the five exports it does have.
# meld's fusion renames them to `pulseengine:falcon-cascade/<stage>@0.7.0#<fn>`.
# * wasmtime's side of the "comparison" invoked that name on a DIFFERENT artifact (the
# pre-fusion component), so the two halves were never looking at the same thing.
# * `2>/dev/null || true` then made "kiln disagreed" and "kiln could not run" the same
# reading — the exact confusion varve#130 recorded as reporting exit 127 as a refusal.
# The JUnit evidence still carried a `kiln-xruntime` testcase whose value was always 0.
#
# WHAT THIS CLAIMS, AND WHAT IT DOES NOT. kiln 0.5.0 (SR-58) can invoke exports on a
# meld-fused CORE module, which is what jess ships. Both engines are driven over the SAME
# artifact, the SAME export and the SAME arguments, and must agree.
#
# It is NOT a value differential, and saying so matters: every cascade export returns an i32
# that is a POINTER into the return area, and that pointer was MEASURED to be constant (9488)
# across three different argument sets. Two engines agreeing on a fixed address is not
# evidence about the arithmetic. What this does establish is that a second, independent engine
# LOADS AND EXECUTES the shipped fused core across all five stage exports without trapping,
# and returns what wasmtime returns. Modest, but true — and it is a real executed result where
# the previous check was a permanently-failing no-op.
#
# Making it a value differential needs kiln to dereference the return area (read N bytes at the
# returned pointer). Filed upstream; until then this check is deliberately labelled down.
set -uo pipefail
ROOT="$(cd "$(dirname "$0")/../.." && pwd -P)"
KILND="${KILND:-kilnd}"
MOD="${MOD:-$ROOT/.scratch/invoke/c.loom.wasm}"
fail() { printf 'FAIL: %s\n' "$*" >&2; exit 1; }

# The export name itself contains colons, so it is NOT safe to pack a field after one.
# A first version wrote "…#mix:4" and split on the FIRST colon, yielding the export name
# "pulseengine" — which then failed as "could not run", correctly, rather than silently
# comparing nothing.
CMP_EXPORT="pulseengine:falcon-cascade/mixer@0.7.0#mix"
ARGS4="0.25 0.5 0.75 0.125"

invoke_kiln() { # $1 export, rest args -> prints the i32 or nothing
local e="$1"; shift
local a=() x
for x in "$@"; do a+=(--arg "$x"); done
"$KILND" "$MOD" --invoke "$e" "${a[@]}" 2>&1 | sed -n 's/.*\[0\] i32 \([0-9-]*\).*/\1/p' | head -1
}
invoke_wasmtime() { local e="$1"; shift; wasmtime run --invoke "$e" "$MOD" "$@" 2>/dev/null | tail -1; }

[ -f "$MOD" ] || fail "fused core not found: $MOD"
command -v wasmtime >/dev/null || fail "wasmtime not on PATH"
command -v "$KILND" >/dev/null || [ -x "$KILND" ] \
|| fail "kilnd not found at '$KILND' — that is 'could not run the check', NOT 'the check failed'"

# kilnd must be >= 0.5.0: earlier builds have no --invoke and cannot reach a fused core at all.
kv="$("$KILND" --version 2>&1 | head -1)"
case "$kv" in
"kilnd "*) : ;;
*) fail "kilnd does not report a version ('$kv'). Pre-0.5.0 builds print a banner instead and
have no --invoke, so this check cannot run against them — refusing rather than reporting
'inconclusive', which is how the previous version hid a permanent failure." ;;
esac
echo " kiln: $kv"

e="$CMP_EXPORT"
k="$(invoke_kiln "$e" $ARGS4)"
w="$(invoke_wasmtime "$e" $ARGS4)"
# BOTH must produce a value. An empty reading is "could not run"; treating it as agreement is
# precisely the defect being fixed here.
[ -n "$k" ] || fail "kiln produced no value for $e — could not run"
[ -n "$w" ] || fail "wasmtime produced no value for $e — could not run"
[ "$k" = "$w" ] || fail "CROSS-RUNTIME DISAGREEMENT on $e: kiln=$k wasmtime=$w"
echo " $e -> kiln=$k wasmtime=$w AGREE"

# And every stage export must at least LOAD AND EXECUTE under kiln. This is the part that is
# genuinely about the shipped artifact rather than about one function.
# Each export is driven with ITS OWN arity. The cascade is not uniform — three distinct
# shapes, measured from the fused module's core signatures:
# ekf#estimate (param f32 x6) -> i32 position/attitude/rate #tick (param i32) -> i32
# mixer#mix (param f32 x4) -> i32
# A first version passed six zeros to all five, so four of them failed on arity and the count
# read 1/5 — a number that says nothing about the artifact. Passing the wrong arity everywhere
# and then reporting the survivors is the shape of a metric that measures the harness.
n=0; total=0; failed=""
for spec in "ekf@0.7.0#estimate:0 0 0 0 0 0" \
"position@0.7.0#tick:0" \
"attitude@0.7.0#tick:0" \
"mixer@0.7.0#mix:0.25 0.5 0.75 0.125" \
"rate@0.7.0#tick:0"; do
stage="${spec%%:*}"; a="${spec#*:}"
full="pulseengine:falcon-cascade/$stage"
total=$((total+1))
if "$KILND" "$MOD" --invoke "$full" $(for x in $a; do printf -- '--arg %s ' "$x"; done) >/dev/null 2>&1
then n=$((n+1)); else failed="$failed $stage"; fi
done
[ "$n" -eq "$total" ] || fail "kiln executed only $n/$total stage exports; failed:$failed"
echo " kiln executed $n/$total stage exports of the shipped fused core"
echo "NOTE: agreement is on a return-area POINTER, measured constant across inputs — this is an"
echo " EXECUTION check, not a value differential (see the header)."
Loading