Fix NOT IN with NULL in the subquery or list to close #1414

[joshuablac]

Closes #1414

Problem

In SQL, x NOT IN (...) is UNKNOWN when x isn't found and the values contain a NULL, so the row is filtered out. AlaSQL returns true instead:

Query	develop	SQL / with this PR
Issue example: a NOT IN (SELECT b FROM S), R = [1, NULL], S = [NULL]	[{a: 1}]	[]
a NOT IN (SELECT b ...) with b = [NULL, 2], a = [1, 2, NULL]	[{a: 1}]	[]
Correlated a NOT IN (SELECT b FROM S2 WHERE S2.id = R2.id) where one row's set is {NULL}	includes that row	excludes it
a NOT IN (2, NULL)	[{a: 1}]	[]
Value of 1 NOT IN (2, NULL)	true	undefined (NULL)

Cause

All four NOT IN code paths in Op.toJS compute plain set membership (!set.has(x) or indexOf(x) < 0): the cached subquery, the correlated subquery, and both literal-list paths (with and without alasql.options.cache). The operator's null check only covers the operands (x), not the values inside the subquery or list.

Fix

A small helper, utils.notIn(set, value), returns:

• false when the value is in the set
• undefined (AlaSQL's NULL) when it isn't, but the set contains NULL (null, undefined or NaN, the same values the operator null check treats as NULL; a NULL inserted into an INT column comes back as NaN)
• true otherwise

The subquery paths (cached and correlated) and both literal-list paths use it. The correlated path now builds a Set like the cached path does. IN keeps its results: when nothing matches it returns false instead of UNKNOWN, but both exclude the row in a WHERE clause.

Cache key for literal lists (affects IN and NOT IN). With alasql.options.cache on (the default), value lists were cached in alasql.sets under values.join(','). Lists such as (2, NULL) and (2, ''), or (1, 3) and ('1', 3), therefore shared one Set. This was already a bug on develop: a IN ('1', 3) run after a IN (1, 3) doesn't match '1'. With this PR's NULL handling it got worse: a NOT IN (2, '') run after a IN (2, NULL) would filter out every row. The key now includes each value's type (JSON.stringify(values.map(v => [typeof v, v]))) and is quoted with JSON.stringify in the generated code, which also avoids broken code for list values that contain a ".

This takes over from #2315, the earlier Copilot attempt, which was closed without merging. That PR ran into the subquery cache, so this one covers both the cached and uncached paths, plus the cached literal-list path.

Not changed here:

• NULL NOT IN (<empty subquery>) is still filtered out, although SQL defines it as TRUE.
• The fallback branch for a right side that is neither a subquery nor a literal list (e.g. a NOT IN @(?) with an array parameter) still uses indexOf, so a NULL in that array isn't treated as UNKNOWN.

Tests

• Added test/test1414.js:
  • the issue's example
  • a subquery with NULL and a value
  • a correlated subquery
  • a literal list with NULL
  • the returned values (undefined / false)
  • a check that NOT IN and IN without NULL in the set are unchanged
  • cached lists that only differ by NULL, '' or type (1 vs '1') are kept apart
• Cases A–E and G fail on develop and pass with this change.
• Full suite (bun test, same command as yarn test-only): 2256 pass, 213 skip. The only failures are the 2 CLI tests in test2149.js, which fail the same way on unmodified develop in my local setup (a space in my checkout path, and the version-injection step of build.sh not running).

This fix was written with AI assistance (Claude Code) and verified locally as described above.

🤖 Generated with Claude Code

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.