Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions .agents/docs/2026-09-21-macros-and-withdrawal-verify.sh
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,99 @@ else
skip "openkal-llvm-runtime 0.14.0 did not resolve from the index"
fi

# ── CHANGE. `builtins = "iso"` withdraws the Apple pattern fill ─────────────
section "G. builtins = \"iso\" emits -fno-builtin (CHANGE)"
# A CODE-GENERATION PROPERTY, so the reading is an object file rather than a
# `-dM` dump — which is exactly why the token this asserts was a silent no-op
# for the whole of its first life. `2026.9.21.2` and earlier emit
# `-fno-builtin-memset_pattern16`; clang matches `-fno-builtin-<fn>` against
# its builtin table, `memset_pattern16` is an LLVM TargetLibraryInfo libfunc
# and not in it, so the flag is accepted in silence and the call survives. On
# those versions the aarch64-macos link fails with
#
# ld64.lld: error: undefined symbol: memset_pattern16
#
# so a failed build here IS the negative reading rather than an absent one.
g="$root/g"; rm -rf "$g"; mkdir -p "$g/src"
cat > "$g/src/main.cpp" <<'EOF'
extern "C" void fill(int* a, long n) {
for (long i = 0; i < n; ++i) a[i] = 0x01020304;
}
int main() { static int buf[64]; fill(buf, 64); return buf[0] == 0x01020304 ? 0 : 1; }
EOF
# `-O2` per package rather than --release: the idiom pass does not run at the
# dev profile's -O0, and a release build would compile the runtime a second
# time in a second profile for no reading.
cat > "$g/mcpp.toml" <<'EOF'
[package]
name = "builtins-probe"
version = "0.1.0"

[build]
cxxflags = ["-O2"]

[dependencies]
openkal-llvm-runtime = "0.15.0"

[toolchain]
default = "llvm@22.1.8"
EOF
gnm=$(ls "$HOME"/.xlings/data/xpkgs/xim-x-llvm/22.1.8/bin/llvm-nm 2>/dev/null | head -1)
if [ -z "$gnm" ]; then
skip "G: no llvm-nm in the payload to read the object with"
elif (cd "$g" && "$STORE" build --target aarch64-macos >/dev/null 2>&1); then
gobj=$(find "$g/target" -name 'main.o' 2>/dev/null | head -1)
if [ -z "$gobj" ]; then
fail "G: the build reported success and produced no object"
elif [ "$("$gnm" -u "$gobj" 2>/dev/null | grep -c memset_pattern16)" = 0 ]; then
ok "builtins = \"iso\" leaves no memset_pattern16 in the object"
else
fail "builtins = \"iso\" did not withdraw memset_pattern16"
fi
else
fail "G: the aarch64-macos build did not complete (the old token's signature)"
fi

# ── CHANGE. `mcpp test --no-run` ────────────────────────────────────────────
section "H. mcpp test --no-run builds the tests and says so (CHANGE)"
# Before this release the flag does not exist and the command exits non-zero
# with "unknown option: --no-run". The runner named here is a program that
# does not exist, which produces "tests built, nothing run" on every host for
# the native target with nothing installed.
h="$root/h"; rm -rf "$h"; mkdir -p "$h/tests"
printf 'int main() { return 0; }\n' > "$h/tests/alpha.cpp"
printf 'int main() { return 0; }\n' > "$h/tests/beta.cpp"
printf '[package]\nname = "norun"\nversion = "0.1.0"\n' > "$h/mcpp.toml"
hhost=$("$STORE" --print-target 2>/dev/null || true)
if [ -z "$hhost" ]; then
(cd "$h" && "$STORE" build >/dev/null 2>&1) || true
hhost=$(ls "$h/target" 2>/dev/null | grep -v '^\.' | head -1)
fi
if [ -z "$hhost" ]; then
skip "H: could not determine the host triple"
else
cat > "$h/mcpp.toml" <<EOF
[package]
name = "norun"
version = "0.1.0"

[target.$hhost]
runner = ["mcpp-no-such-runner-exists"]
EOF
hout=$( (cd "$h" && "$STORE" test --target "$hhost" --no-run 2>&1) )
hrc=$?
case "$hout" in
*"2 built, not run"*)
if [ "$hrc" = 0 ]; then
ok "--no-run builds the tests and exits 0"
else
fail "--no-run reported the built tests and exited $hrc"
fi ;;
*) fail "--no-run did not report two tests as built" ;;
esac
fi


printf '\n-- summary --\nfails=%d\nnot run:%s\n' "$fails" "${skipped:-
(none)}"
[ "$fails" -eq 0 ]
216 changes: 214 additions & 2 deletions .agents/docs/2026-09-21-openkal-ecosystem-completion-and-acceptance.md

Large diffs are not rendered by default.

123 changes: 123 additions & 0 deletions .github/workflows/openkal-cross.yml
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,129 @@ jobs:
echo "${{ matrix.host }} → $t : $(ls -l "$a" | awk '{print $5}') bytes"
done

# THE ONLY CRITERION THE `builtins` TOKEN HAS, AND WHY IT IS HERE.
#
# `[c-abi] builtins = "iso"` states that the C library supplies the ISO
# functions and no vendor extensions. On Darwin targets clang's loop
# idiom recogniser rewrites a constant-pattern fill into a call to
# `memset_pattern16`, an Apple libc extension no such library carries.
# That call is produced by the code generator, so it appears in no `-D`
# and in no preprocessor dump --- and `mcpp.toolchain.cenv` verifies its
# tokens by comparing a `-dM` dump. The token this mechanism emitted was
# therefore a silent no-op for the whole of its first life, and this
# step exists because no other kind of check could have reported it.
#
# LEG 1 IS THE DENOMINATOR, AND IT IS NOT DECORATION. At `-O0` the pass
# does not run, and `-ffreestanding` implies `-fno-builtin`; an
# assertion that only reads "the symbol is absent" therefore passes in
# several worlds where nothing was measured. Leg 1 compiles with no flag
# at all and fails if the symbol does NOT appear.
#
# LEG 2 PINS THE DEFECT ITSELF. `-fno-builtin-memset_pattern16` is
# accepted in silence and changes nothing: `-fno-builtin-<fn>` is matched
# against clang's builtin table, and `memset_pattern16` is an LLVM
# TargetLibraryInfo libfunc rather than a clang builtin. The toolchain is
# pinned in this job, so the reading is stable. Should a later pin make
# leg 2 fail, clang has gained the narrower behaviour, and `cenv` can
# emit the narrower flag and recover the 1.8 per cent `-fno-builtin`
# costs.
#
# LEG 3 IS THE ENGINE: the same idiom, over the openkal stack, for
# `aarch64-macos`, built by the mcpp under test. Measured on the token
# this step was written for, the two readings are
#
# -fno-builtin-memset_pattern16 1 reference, and the link fails
# -fno-builtin 0 references, and it links
#
# so a regression here reports itself at the link before the assertion
# is reached. The assertion covers the remaining case, in which some
# layer happens to supply the symbol and the link succeeds anyway.
#
# `-O2` IS PER PACKAGE RATHER THAN `--release`. The idiom pass does not
# run at the dev profile's `-O0`, and a release build would compile the
# runtime a second time in a second profile for no reading.
- name: builtins = "iso" withdraws the Apple pattern fill
run: |
set -euo pipefail
BIN="${MCPP_HOME:-$HOME/.mcpp}/registry/data/xpkgs/xim-x-llvm/22.1.8/bin"
# Not `ls ... | head -1`: with `pipefail` the absent candidate's
# exit status ends the step before the guard below is reached.
pick() { # $1..$n = candidate paths; prints the first executable one
for c in "$@"; do
if [ -x "$c" ]; then printf '%s' "$c"; return 0; fi
done
return 1
}
CLANG="$(pick "$BIN/clang" "$BIN/clang.exe")" \
|| { echo "::error::no clang under $BIN"; exit 1; }
NM="$(pick "$BIN/llvm-nm" "$BIN/llvm-nm.exe")" \
|| { echo "::error::no llvm-nm under $BIN"; exit 1; }

# Inside the clone, so the dependency is named by a relative path.
# `$RUNNER_TEMP` is a backslash path on the Windows host and a TOML
# string would read its separators as escapes.
W="$RUNNER_TEMP/okl/examples/builtins-probe"
rm -rf "$W"; mkdir -p "$W/src"

# The one shape the idiom recogniser rewrites. The element type is
# `int` and not `char` because a byte-repeating value becomes
# `memset`, which every C library has.
cat > "$W/src/main.cpp" <<'PROBE'
extern "C" void fill(int* a, long n) {
for (long i = 0; i < n; ++i) a[i] = 0x01020304;
}

int main() {
static int buf[64];
fill(buf, 64);
return buf[0] == 0x01020304 ? 0 : 1;
}
PROBE

cat > "$W/mcpp.toml" <<'PROJECT'
[package]
name = "openkal-builtins-probe"
version = "0.1.0"

[build]
cxxflags = ["-O2"]

[dependencies]
openkal-llvm-runtime = { path = "../.." }

[toolchain]
default = "llvm@22.1.8"
PROJECT

# THE READER ASKS FOR UNDEFINED SYMBOLS, NOT FOR BYTES.
#
# This was `grep -ac memset_pattern16`, on the reasoning that the name
# is in the object's string table and a byte match needs no tool. It
# reads 1/0 correctly with GNU grep and read 0 for all three legs on
# the macOS host, where grep is BSD: the object is binary, and what
# `-a` promises about that differs between the two. Leg 1 is what
# reported it. `llvm-nm` is in the payload beside the clang already
# being used, and answers the question this step is actually asking.
refs() { "$NM" -u "$1" 2>/dev/null | grep -c memset_pattern16 || true; }
T=--target=arm64-apple-macos14.0

"$CLANG" $T -O2 -c "$W/src/main.cpp" -o "$W/bare.o"
"$CLANG" $T -O2 -fno-builtin-memset_pattern16 -c "$W/src/main.cpp" -o "$W/narrow.o"
"$CLANG" $T -O2 -fno-builtin -c "$W/src/main.cpp" -o "$W/blunt.o"
echo "no flag : $(refs "$W/bare.o")"
echo "-fno-builtin-memset_pattern16 : $(refs "$W/narrow.o")"
echo "-fno-builtin : $(refs "$W/blunt.o")"

[ "$(refs "$W/bare.o")" = 1 ] || { echo "::error::leg 1: the probe no longer triggers the idiom, so legs 2 and 3 measure nothing"; exit 1; }
[ "$(refs "$W/narrow.o")" = 1 ] || { echo "::error::leg 2: clang now honours -fno-builtin-memset_pattern16, and mcpp.toolchain.cenv can emit the narrower token"; exit 1; }
[ "$(refs "$W/blunt.o")" = 0 ] || { echo "::error::leg 2: -fno-builtin no longer withdraws the pattern fill"; exit 1; }

(cd "$W" && "$MCPP_UNDER_TEST" build --target aarch64-macos)
obj=$(find "$W/target" -name 'main.o' | head -1)
[ -n "$obj" ] || { echo "::error::leg 3 produced no object to read"; exit 1; }
echo "engine, aarch64-macos over openkal: $(refs "$obj")"
[ "$(refs "$obj")" = 0 ] || { echo "::error::leg 3: builtins = \"iso\" did not withdraw memset_pattern16"; exit 1; }

- uses: actions/upload-artifact@v4
with:
name: openkal-built-on-${{ matrix.host }}
Expand Down
108 changes: 107 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,113 @@
> 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。
> 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。

## [Unreleased]
## [2026.9.21.3] - 2026-09-21

### `mcpp test --no-run`:为一个跑不了的目标构建测试,并把这当成答案

一个本机既不能执行、也没有 runner 可达的目标,会让每个测试停在 `not run`,命令退出 2。
这是对「这些测试通过吗」的正确回答——mcpp 没有查明。但 **2 同样是 runner 坏掉时的
退出码**,于是一个只想要「构建」的调用方无法区分这两者,只能退回去用 `mcpp build`。

而 `mcpp build` 构建的是**包**。对一个源码只在 `tests/` 下的包,它一行都不编译。
实测 mcpp-index 的 `archive` 成员(源码是 `tests/` 下两个文件):

```
$ mcpp build --target aarch64-macos # 退出 0
Compiling compat.lz4 / compat.xz / compat.zlib / compat.zstd ...
$ find target -name '*compression*' -o -name '*versions*'
(只有 musl 的 versionsort.o)
```

退出 0,编译了这个成员的**依赖**,而**该成员自己的代码一行都没有编过**。一个读这个
退出码的兼容性测量,会把它记成「这个成员在 macOS 上构建通过」。

`--no-run` 让那个更窄的断言有了自己的答案:被选中的每个测试都为该目标编译并链接,
没有任何一个被执行,结果也这么写:

```
test result ok. 0 passed; 0 failed; 2 built, not run
```

`built` 与 `not_run` 分开计数,机器接口也一样(`docs/50`):`not_run` 是「试过而做不到,
问题悬着,退出 2」,`built` 是「被要求不要执行,构建就是问题的全部,退出 0」。编译不过
的测试仍然是失败。`--no-run` 与 `--no-runner` 同时给出会被拒绝——两个名字只差一个字符
而含义相反,不存在应当优先的那一个读法。

判据是 `tests/e2e/745_no_run_builds_the_tests_and_says_so.sh`,四条腿。其中 runner 用的是
一个**不存在的程序名**:用一个执行不了的目标会让这个测试需要交叉工具链和特定宿主,而
一个找不到的 runner 在每台宿主上、对本机目标、不装任何东西,就能造出同一个局面。

### `builtins = "iso"` 发的那个 token 是静默空操作,已换成 `-fno-builtin`

`[c-abi] builtins = "iso"` 声明 C 库只提供 ISO 函数、没有厂商扩展。Apple 目标上
clang 的循环惯用法识别会把常量模式填充改写成 `memset_pattern16` 调用——那是一个
Apple libc 扩展,这样的库没有它。此前这里发的是 `-fno-builtin-memset_pattern16`。
在一份真实的 `build.ninja` 编译命令上做 A/B,只改这一个 flag:

| flag | `memset_pattern16` 引用数 |
|---|---|
| 照原样(flag 在) | 1 |
| flag **删掉** | 1 |
| `-fno-builtin` | 0 |
| `-mllvm -disable-loop-idiom-memset` | 0 |

**而它报不出自己什么都没做。** clang 静默接受 `-fno-builtin-totally_not_a_function`:
`-fno-builtin-X` 这一族按 clang 的 builtin 表校验,而 `memset_pattern16` 是 LLVM
TargetLibraryInfo 的 libfunc,不在那张表里;发出调用的是 LoopIdiomRecognize,它查
TLI,按函数名的属性到不了它。

不选 `-mllvm` 的理由是它传的是 LLVM 内部选项,不是受支持的接口,改名或删除之后这套
机制会再次静默失效——那正是这次要修的缺陷本身。代价是量出来的:在暴露此事的那个翻译
单元(libarchive 的 7zip reader,`-O2`,aarch64-macos)上,目标文件从 38200 涨到
38888 字节,1.8%,因为 `-fno-builtin` 同时撤走了 C 库确实提供的那些 ISO 函数。这比
`builtins = "iso"` 声明的范围宽,而它宽在安全的方向:代码生成器不合成的调用不会变成
链接错误。

**这个空操作能活下来,是因为它没有判据。** `cenv` 的探针用 `-dM` dump 校验自己发的
token,而代码生成阶段的性质在预处理器 dump 里不可见。判据现在在
`.github/workflows/openkal-cross.yml`,三条腿,跑在三台宿主上:

| 腿 | 内容 | 判据 |
|---|---|---|
| 1 | 不带任何 flag 编译探针 | 符号**必须出现**——否则探针已经触发不了惯用法,腿 2、3 什么都没测 |
| 2 | `-fno-builtin-memset_pattern16` | 符号**仍必须出现**(钉住这个缺陷;若哪天红了,说明 clang 认了窄拼法,`cenv` 可以改回去把这 1.8% 拿回来) |
| 3 | mcpp 为 `aarch64-macos` 走 openkal 栈构建同一份源码 | 目标文件里**零引用** |

腿 3 的对照:换回旧 token,同一个工程链接失败于
`ld64.lld: error: undefined symbol: memset_pattern16`。

### 更正:那个「四个成员」是二,而分组用错了依据

2026.9.21.2 的条目、`cenv.cppm` 与 `predefines.cppm` 的注释、`docs/21` 与 `docs/22`
都写着「借来的 `__CYGWIN__` 代价是四个成员:archive、sqlite3、mimalloc、c-ares」。
**撤销之后的重测把这个数目否掉了。**

| 成员 | 记的真因 | 实测真因 | 撤销后 |
|---|---|---|---|
| `archive`(经 xz) | `__CYGWIN__` | **对** | **清了** |
| `sqlite3` | `__CYGWIN__` | **对** | **清了** |
| `c-ares` | `__CYGWIN__` | **错**:`#ifdef HAVE_WINDOWS_H`,而那个宏由 mcpp-index 自己的配方在 windows 分支 `#define` | 仍红 |
| `mimalloc` | `__CYGWIN__` | **错**:已经不走到任何头文件——`error in backend: Target OS doesn't support __builtin_thread_pointer() yet` | 仍红 |

**四个是按「诊断」分的组,不是按「真因」。** 四个都停在 `windows.h`(mimalloc 当时如此),
于是被记成同一类。**按诊断分组不是按真因分组**,这样数出来的数目会高估一次撤销能修掉多少。

这条更正本身来自判据:重测把总失败从 10 降到 5、**新增失败为零**,而「`windows.h` 组 4→0」
这半条没有达成——是 4→2。**一个达成了一半的判据,比一个没写的判据更有价值:它指出了
分母是错的。**

### `mimalloc` 暴露出替身三元组的一个代价,与宏无关

```
fatal error: error in backend: Target OS doesn't support __builtin_thread_pointer() yet.
```

`presents = "posix"` 在 Windows 上实现成 `--target=x86_64-pc-cygwin`。LLVM 没有为那个
OS 实现 `__builtin_thread_pointer()`,而 mimalloc 用它取线程局部堆指针。**这是替身
三元组的第一个被测量到的、超出宏名之外的代价**;先前关于这次替换的记录只讨论了预处理器
看到什么。


## [2026.9.21.2] - 2026-09-21

Expand Down
18 changes: 18 additions & 0 deletions docs/08-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ mcpp performs with the test binary appended.
```bash
mcpp test --target thumbv7em-none-eabihf # built for the board, run through its runner
mcpp test --no-runner # ignore the runner and execute directly
mcpp test --target aarch64-macos --no-run # build the tests for the target and stop
```

Nothing about the test changes. The same `tests/**/*.cpp` compiles for the
Expand All @@ -114,6 +115,23 @@ or a QEMU exit code is what a bare-metal runner is chosen to produce.
`--no-runner` exists for a host that can execute the binaries natively and
should not pay for an emulator.

`--no-run` makes the narrower claim, and it has to be asked for. Without it, a
target this host can neither execute nor reach through a runner leaves every
test not run and the command exits 2: mcpp did not establish whether the tests
pass, and reporting that as success is the false reading this repository has
recorded most often. But 2 is also what a broken runner returns, so a caller
that wanted only the build could not tell the two apart. Under `--no-run` every
selected test is compiled and linked for the target, none is executed, and the
result says so:

```
test result ok. 0 passed; 0 failed; 2 built, not run
```

A test that does not compile is still a failure, and `--no-run` with
`--no-runner` is refused rather than resolved: one says to run the binaries
without the declared runner, the other says not to run them.

A test program carries the files it reads beside it: the runner receives
`MCPP_RUNTIME_FILES`, the list of its deployed files and the shared libraries it
loads, and a runner that moves the program to a device copies them with it. A
Expand Down
Loading
Loading