Repository navigation
Check every public function against cross-platform reference values - #78
Merged
Merged
Conversation
FirebladeMath should produce identical results on every platform, up to rounding. Until now nothing verified that across platforms: the differential tests compare portable vs simd only on Apple platforms, and the C math libraries (Apple libm, glibc, ucrt, musl/WASI) were never compared at all. All 241 public functions (top-level functions plus the static functions and initializers of the matrix and quaternion types; generics in Float and Double, and eulerAnglesXYZ) are registered with an accuracy class and input domains. `make reference-values` evaluates them on 24 deterministic inputs each (plus NaN, ±inf, ±0, ±1, tiny and huge values for scalar functions) on the reference platform (macOS, default traits) and stores inputs and results as exact bit patterns in ReferenceData.swift. Every platform that runs the tests compares its results against them: - exact: sqrt, floor, copysign, fmod, predicates, transpose, clamp, lerp, orthographic, ... must be bit-identical - libm: single C math library calls within 4 ulp - composite: rounding-level relative tolerance (products scale with the inputs, loose for inverses, trigonometry, Euler angles, projections) - NaN, infinities and the sign of zero must always match exactly Scripts/check-reference-coverage.py fails if a public function is not registered or a registration is stale. The WASI job now runs the tests instead of only building them. Claude-Session: https://claude.ai/code/session_011WSUtoLptjea3s3iC5wTNC
Running the tests on WASI (new in this PR) showed two test problems: - The pseudo-random input generators computed `Double(1 << 53)`. Int is 32-bit on wasm32, so the shift produced 0 and every generated value became infinity (512 failed expectations). Multiply by 0x1p-53 instead. - FunctionTests compared C math library results (tanh, sin, pow, ...) for exact equality. WASI's libc returns tanhf(0.5) 1 ulp away from Apple's libm, which is legitimate. These 33 spot checks now allow 4 ulp, matching the libm class of the reference-value tests; exact operations stay exact. The reference-value test itself passed on WASI. Claude-Session: https://claude.ai/code/session_011WSUtoLptjea3s3iC5wTNC
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes "identical up to rounding on every platform" a CI-checked guarantee instead of an assumption.
#77 compares FirebladeMath's portable implementations against Apple's simd, but only on Apple platforms, and only for functions that have both. The C math libraries the wrappers call (Apple libm, glibc, ucrt, musl/WASI) were never compared at all.
How it works
ReferenceFunctions.swift): all 241 public functions, each with an accuracy class and input domains. That's every top-level function plus the static functions and initializers of the matrix and quaternion types. Generic functions are registered inFloatandDouble, andeulerAnglesXYZis included too.ReferenceData.swift):make reference-valuesevaluates every function on macOS (default traits) and stores inputs and results as exact bit patterns. The data is one raw string, so it adds no compile-time cost.exactsqrt,floor,copysign,mod, predicates,transpose,clamp,lerp,orthographic, …libmsin,pow,atan2, …)composite(.products)dot,cross,multiply,actcomposite(.tight / .loose)Scripts/check-reference-coverage.py(make check-reference-coverage, in the macOS CI job) fails if a public function isn't registered, or if a registration is stale.Platforms: CI results
swift testunder WasmKitSo "identical up to rounding" holds on every platform that can run tests: exact operations are bit-identical, C-library calls are within 4 ulp, and NaN, ±∞ and the sign of zero match exactly.
Test fixes found by running on WASI:
Double(1 << 53). On wasm32Intis 32 bits, so that was 0 and every generated value became ∞. They now use* 0x1p-53. FirebladeMath itself was not affected.FunctionTestsspot checks compared C-library results for exact equality. WASI'stanhf(0.5)is 1 ulp away from Apple's, which is legitimate. Those checks now allow 4 ulp, matching thelibmclass; exact operations stay exact.Verified locally
macOS with simd and portable (trait off): all 248 registrations match, so FirebladeMath's own code reproduces the simd-generated values.
Mutation check: I injected six bugs across the accuracy classes:
fractNaN handling;lerpformula;perspectiveRHhalf-angle factor;pow2base;cross.All six are caught, each naming the affected functions.
Coverage check: removing one registration fails the check and names the function.
CI results for Linux, Windows and WASI follow below.
https://claude.ai/code/session_011WSUtoLptjea3s3iC5wTNC