[CaffeinatedCoder]

Fixes #257

Supersedes #282, whose head branch lives on EntglDb and can no longer be updated by the author. This is #282's commit rebased onto main (#252 is merged), plus one commit for the review item on #282.

What was wrong

IndexKey(double) gave a NaN with the sign bit clear a key above +Infinity, and a NaN with the sign bit set (the constant double.NaN) a key below -Infinity (#257). Every computed NaN on Arm64 (0.0 / 0.0, Math.Sqrt(-1), ∞ - ∞) has the sign bit clear; on x64 it has it set. So index-served orderings put those documents last on one CPU and first on the other, and index-served counts with a lower bound (Count(x => x.Ratio > 0.5) through CountRange) included them.

Mechanism

The sort-preserving encoding (IndexKey.cs) flips all bits of a value with the sign bit set and only the sign bit otherwise. A NaN's sign bit and payload are arbitrary, so the two NaN halves landed below -Inf and above +Inf, and every payload got a distinct key.

Fix

Every NaN now takes one key: that of the bits 0xFFF8000000000000 (double.NaN), spelled out as a constant so the key does not depend on how the NaN was produced. That key sorts below -Infinity, which is where double.CompareTo (and so LINQ-to-Objects and the unindexed path) puts NaN. A range with a finite lower bound no longer reaches it.

Range semantics: NaN is now the smallest double key, as -Infinity was before. < x served from the index includes NaN, the same way it includes -Infinity. Whether < x should exclude NaN (C# comparison operators say false) is the open question the issue names; the BSON post-filter decides for Where, not for Count. I left that as it is.

Review item from #282: migrating keys already stored

Positive-NaN keys already stored (on Arm64) sat under the old above-+Infinity key with no migration, so lookups missed them and deleting the document left a stale entry. NeedsKeyFormatMigration now also looks for any key above that of +Infinity in a double- or float-keyed index, the same way #252 looks for the former -0.0 key, and such an index is rebuilt on open. This costs one more seek per index and open. No KeyFormat bump: the encoding of every non-NaN double is unchanged.

Tests

IndexKeyNaNTests:

• Every_NaN_Has_The_Key_Of_Double_NaN_Below_Negative_Infinity, a theory over a positive NaN (0x7FF8…), a positive payload NaN, a negative payload NaN and double.NaN. Each must equal IndexKey.Create(double.NaN), sort below -Infinity and MinValue, and round-trip to a NaN. Four of the five cases fail without the fix; the double.NaN row already had this key.
• Index_Orders_NaNs_First_And_Excludes_Them_From_A_Lower_Bounded_Range, on BTreeIndex: ascending order is NaNs first, then 1.0 and +Inf; >= 0.5 returns 1.0 and +Inf only; >= double.MaxValue returns +Inf only.
• Index_Holding_The_Former_Positive_NaN_Key_Is_Rebuilt_On_Open (new): writes the former key of a positive NaN into a double index, reopens, and checks the index was rebuilt with the canonical NaN key, that deleting the document removes its entry, and that an index holding +Infinity but no former NaN key is not rebuilt. It fails without the migration change.

dotnet test tests/BLite.Tests on the #282 base (with #252): 3104 passed, 0 failed, 4 skipped. On main after the rebase: 3176 passed, 1 failed, 4 skipped. The failure was IndexPageReleaseTests.CreateDropIndexCycles_Typed_DoNotGrowTheFile, which passed 3 of 3 times when run on its own. A second full run on a heavily loaded machine (load average 28) failed in 7 unrelated tests, all with Timed out acquiring PageFile write lock, a different set each run (#303).

Compatibility

• DynamicCollection double keys (DynamicCollection: range filters and sorts on an indexed double field return wrong results (keys are little-endian BitConverter bytes) #254) do not go through IndexKey(double) today. If DynamicCollection: range filters and sorts on an indexed double field return wrong results (keys are little-endian BitConverter bytes) #254 is fixed that way, they inherit this.
• A negative NaN with a payload other than double.NaN's also had a key apart from the canonical one. Such NaNs only come from explicitly constructed bits, and the migration does not look for them.

The B-tree model test PR (#259) lists this as a known candidate. Its registry entry can go once this lands.

🤖 Generated with Claude Code