feat(sql,core): story 22.6 — set operators (UNION / UNION ALL / UNION DISTINCT / INTERSECT [ALL] / EXCEPT [ALL]) - #356
Merged
Merged
Conversation
…LL] / EXCEPT [ALL] Story 22.6. `UNION ALL` was the only spelling the grammar accepted; the bare, de-duplicating `UNION` was a parse error and `INTERSECT` was read as a table ALIAS. All six SQL set operators now parse into one flat AST, are validated for branch arity and type compatibility before any client is touched, and route to the relational engine — while the `UNION ALL` fast path does not move a byte. AST: `MultiSearch` gains `operators: Seq[SetOperator]`, appended LAST and defaulted. `Nil` is the CANONICAL form for a `UNION ALL`-only list, not merely a default: without that normalisation a programmatic `MultiSearch(reqs)` and its own re-parse render identically and compare UNEQUAL, so the house fixed point `Parser(stmt.sql) == Right(stmt)` would fail for the one shape that pre-dates this story. Readers consult `resolvedOperators`. Precedence (INTERSECT tighter, else left-associative) is the DERIVED `tree`; parentheses are never recorded. Routing: ONE predicate. `MultiSearch.relationalClosureRequired` reads the branch's STATEMENT member, so 22.3's `hasCorrelatedSubqueries` and 22.5's `ctes.nonEmpty` flow through it unchanged. The package function is refactored over a single `embeddedSearchStatements` arm list so the set-operation level is visible without a second `match`. Guards, all reading that one predicate: the `resolveWithSchema(multiple)` seam (which also RE-RUNS the type check once schemas are attached — DuckDB casts implicitly across branches, so the sql side is the only type guard), `CoreDqlExtension`, the `searchAs` macro, `CreateMaterializedView.validate()`, and a NAMED `require` backstop in both bridge copies. Render: `UNION ALL`-only is byte-for-byte today's text; anything else parenthesises every branch — and branch 0's `WITH` prefix is HOISTED out of the parentheses, because a branch is a `single` and `single` has no WITH. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…th them Story 22.6 AD-6 — issue #209's family, one venue over, and a lead-approved fold-in. Its own commit so it can be split without re-deriving. `SearchApi.search(MultiSearch)` built ONE `_msearch`, and an `_msearch` sends every leg as its own ONE-SHOT request: a row-shaped leg with no bounding LIMIT came back with Elasticsearch's default 10 hits, HTTP 200, no truncation flag. MEASURED on real ES 8.18 over two 3-shard indices holding 30 and 24 documents: 54 rows expected, 20 returned. This story pins this path as "the fast path", and pinning a silently truncating request byte-for-byte would pin a defect as a contract. Such a statement now executes per leg through `search(leg)` — which routes each leg through scroll/PIT exactly as that leg executes on its own, so the schema attach, the temporal literals and the #224 error translation apply per leg BY CONSTRUCTION — and the legs are concatenated in order. Sequential, not parallel: N un-LIMITed legs in parallel would open N scrolls for a consumer that concatenates them anyway. Every other shape (every leg bounded within `max_result_window`, or aggregation-shaped) keeps the single `_msearch`, byte for byte. 🔴 The licensed cap HAD to move with the routing. `CoreDqlExtension` executed a `MultiSearch` uncapped, which was harmless only because `_msearch` truncated every un-LIMITed leg to 10 rows anyway. New `capUnionAll` applies ADR D4's rules to the CONCATENATED total: (1) a branch LIMIT above a finite quota is a 402 before anything executes; (2) a row-shaped un-LIMITed branch under a finite quota runs the legs sequentially with a RUNNING budget — leg i is bounded by what the earlier legs left, a spent budget skips the remaining legs entirely, and exactly ONE cap-hit is recorded for the statement; (3) everything else executes unchanged. Rule (2) carries `!ResultCapContext.isSuppressed`, so a derived leg whose body is a `UNION ALL` is not capped inside the engine.⚠️ Release notes: an un-LIMITed row-shaped `UNION ALL` now returns EVERY row (it returned 10); under the licensed cap it answers `QueryRows` where it answered `QueryStructured`; and within one leg row order may differ, because an un-ordered extraction is interleaved across sliced PIT readers (#238). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Closed Issue #353
One HIGH and six MEDIUM findings from the independent review of `feature/22.6`, each reproduced against a control at `b7f40955`. 🔴 F1 — a silent wrong answer this branch INTRODUCED, inside the commit written to close one. `cappedUnionAllRows` scrolled EVERY row-shaped leg, and `ScrollApi.scroll(statement, config)` does not honour the statement's own LIMIT (pre-existing, and latent until this made it reachable). MEASURED on real ES 8.18 over 30- and 24-document indices, `SELECT id FROM a LIMIT 3 UNION ALL SELECT id FROM b`: SQL says 27, `main` returned 13 (the #209 truncation), this branch returned 54 — leg 1's LIMIT gone, QueryRows, HTTP 200, `truncated = false`. Only an UN-LIMITED row leg is scrolled now; every other leg is executed and bounded by the budget. No test could see it: the budget rows all used un-LIMITed legs, and the completeness spec's mixed row goes through `client.search`, the route where it always worked. F2 — the per-leg route changed the ROW SHAPE of a heterogeneous `UNION ALL`: the one-shot path keys every leg by the FIRST branch's output names, the per-leg path kept each leg's own, so only the presence of a LIMIT decided whether a consumer got ragged rows. The merge now normalises positionally, as the one-shot path does. F3 — the "never truncate silently" row could not fail for the reason it named (the injected failure took a different arm). The failure is now injected at the client, and the executor's result-variant contract is pinned directly — which REFUTES the review's sub-claim that `QueryStream` becomes reachable: measured, `dqlExecutor.execute` collects the stream and answers `QueryStructured` for every shape the fold receives. The arm stays as defence in depth and the pin says so. F4 — AD-4's seam type re-check was untested (disabling it left core 1093/1093 green). Pinned where a stub schema exists, with a control. F5 — the WHERE-subquery rejection hard-coded "UNION ALL", newly reachable for the five other spellings, so it named the operator the analyst did NOT write. It now names theirs; `select.json`'s limitation line follows. F6 — the cap-hit fired before anything ran, so a `UNION ALL` that fitted inside the quota recorded a quota hit beside `truncated = false` — the meter and the flag contradicting each other on one statement (extensions#46's class). It now fires only when the cap bit, and the warning is empty when it did not. Also: the msearch byte pin renders each leg's own SQL, so it can tell a correct `_msearch` from a wrong one instead of comparing two `match_all`s; the row-shape assertion no longer loops over zero rows; rule (3)'s comment stops claiming an aggregate cap it does not apply. 8 mutations re-falsified, 8 RED. `UnionAllCompletenessSpec` re-checked against the control: still 4/6 RED (20 vs 54, 15 vs 29). Closed Issue #353 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --- RELEASE NOTES for story 22.6 (PD-7 i-xi) — recorded on the branch, not only in the spec. The two earlier commits are already pushed and reviewers cite them by hash, so the list rides here rather than in an amended history. (i) `MultiSearch` arity 2 -> 3 (`operators`, defaulted last) — BINARY-INCOMPATIBLE; a downstream `case MultiSearch(a, b)` must add an arm. (ii) `intersect` is now a RESERVED word — a bare column or alias named `intersect` must be quoted (`"intersect"`). The story's one narrowing; zero occurrences in this repo's own 1,408-statement corpus. (iii) NEW operators `UNION [DISTINCT]`, `INTERSECT [ALL]`, `EXCEPT [ALL]` parse and execute at engine venues, and are refused with HTTP 400 naming `softclient4es-arrow-extensions` elsewhere — a bare `UNION` that was a parse error is now a statement. (iv) A trailing `ORDER BY` / `LIMIT` after the last branch of a non-`UNION ALL` set operation is rejected, naming the derived-table spelling. Never parsed before, so no regression. (v) `UNION ALL` legs that are row-shaped with NO LIMIT now return EVERY row (they returned 10 — the #209 truncation), and the licensed `maxQueryResults` cap applies to the concatenated TOTAL. (vi) 🔴 BRANCH ARITY / TYPE MISMATCHES IN `UNION ALL` ARE NOW PARSE-TIME 400s. This BREAKS PREVIOUSLY-EXECUTING STATEMENTS: `SELECT a, b FROM t UNION ALL SELECT a FROM u` and a BIGINT-vs-VARCHAR branch pair used to run and return ragged / silently-coerced rows. The documentation promised the check as "⚠️ not implemented yet"; it is implemented now. (vii) arrow: `JoinPlan` arity 5 -> 6 (`setOperation`); `JoinPlanner.plan` PLANS a `UNION ALL` with cross-index JOIN branches where it used to refuse; the `SelectStatementPattern` anchor tolerates a leading `(`. (viii)`SELECT * EXCEPT(cols)` projection exclusion is untouched. (ix) A LONE PARENTHESISED SELECT `(SELECT … FROM …)` now parses (it was a parse error); a whole set operation wrapped in parentheses is rejected naming the unwrapped spelling. (x) On the licensed gateway an un-LIMITed `UNION ALL` capped by quota now answers `QueryRows` (was `QueryStructured`), and within one leg row order may differ from before (per-leg scroll, #238 slicing). (xi) `CREATE MATERIALIZED VIEW` over ANY set operation (including `UNION ALL`) is a parse-time 400 where it used to throw an internal error.
…N ALL route
The delta review of `f64f475d` came back BLOCK. Findings 1, 2, 4 and 5 are one
question — what is the row contract of a `UNION ALL`, and does every route honour
it — and the lead's instruction was to decide it once and apply it where all three
routes pass through, rather than patch `mergeLegResponses` again. That is what this
does.
🔴 THE CONTRACT: `ElasticConversion.rowNormalizer` over the FIRST branch's output
names — a BY-NAME lookup that null-fills a miss and appends an extra — hoisted once
per statement and applied by the per-leg route and by the licensed cap fold. The
one-shot `_msearch` route already applies exactly that function.
1 — my positional re-key changed which VALUE sat under a column name, and its
stated premise was false. I wrote that positional "is what the one-shot path does";
it is not — `multiSearch` hands `requests.head`'s names to `parseResponseTree`,
which applies `rowNormalizer`, by NAME. Measured on real ES 8.18, `SELECT category,
tag FROM l UNION ALL SELECT tag, category FROM r` (legal: same arity, same type)
came back with `category` holding the TAG:
route before after
one-shot {category -> R_CAT_1, tag -> R_TAG_1} unchanged
per-leg {category -> R_TAG_1, tag -> R_CAT_1} {category -> R_CAT_1, tag -> R_TAG_1}
2 — a `SELECT *` leg was worse. `MultiSearch.declared` is `None` for `SELECT *`, so
`extractOutputFieldNames` returns nothing and the leg's rows arrive in `_source`
order; `zip` put the id under `category` and TRUNCATED the row to the first
branch's width, dropping two columns with HTTP 200. Measured after: both routes
return `{category -> R_CAT_1, tag -> R_TAG_1, id -> R_id_1, amount -> 1}`.
4 — the cap fold concatenated leg rows with no re-keying at all, so ONE statement
had THREE row shapes depending on route, and the licensed gateway — the surface the
REPL, JDBC and Flight use — had the ragged one. It now applies the same hoisted
normaliser, and a test compares its rows against the un-capped route's on the same
statement and the same fixture.
5 — AD-4's cross-branch TYPE re-check lives in `resolveWithSchema(multiple)`, which
the cap fold never called: `SELECT category … UNION ALL SELECT amount …` answered
400 through `client.search` and HTTP 200 with 54 rows through the licensed gateway.
The fold consults the seam first, before any leg runs.
3 — 🔴 MY REFUTATION WAS FALSE. I recorded, in the code and in a commit message,
that `dqlExecutor.execute` collects every stream and answers `QueryStructured`, so
the `QueryStream` arm was unreachable. `DqlRouterExecutor` routes on
`limit.isDefined || fields.isEmpty`, and `fields` is empty only under GROUP BY or
windowing — so a leg that is not row-shaped, carries no LIMIT and projects fields
(`SELECT a, MAX(a) AS m FROM t`) IS answered as `QueryStream`. The probe that
"measured" the opposite only ever asked about shapes satisfying that condition. The
arm now materialises the stream bounded by the budget, and its pin computes the
expected variant from the ROUTER'S OWN condition per statement and asserts both
variants really occur.
6 — `renamed` allocated `row.keys.toSeq` and rebuilt a `ListMap` per row, on the
route taken precisely when a leg has no LIMIT. The decision is leg-constant and is
now hoisted, which is what `rowNormalizer` was built for.
7 — `rows.size >= max` cannot tell "exactly `max` rows exist" from "the result was
cut". Each leg is now ASKED for one row more than it may contribute and keeps at
most the budget; raising the budget instead would have been the obvious edit and a
regression, since `acc` would reach `max + 1` and a later leg would still have
budget left — losing the property that legs past a spent budget never run.
8 — MUT-1, the prior review's silent-drop mutation, still SURVIVED the full core
suite: nothing reached the `case other` arm. A stub executor answering `EmptyResult`
now does, and the message names the inner variant instead of `ElasticSuccess`,
which is what it reported for every possible value.
Two vacuous assertions repaired. Re-falsified every mutation from both reviews and
both delta rounds: 25/26 RED, MUT-1 included; the one GREEN is an arrow lookahead
the review proved redundant. sql 1325 · core 1107 · `+ core/compile` ·
`++ 2.12.20 core/Test/compile`. Real ES 8.18: UnionAllCompleteness 6/6, REPL 78/78.
⚠️ One shape where the routes still differ, measured rather than assumed: for
`SELECT id AS x FROM l UNION ALL SELECT id AS y FROM r` the one-shot route answers
`{x -> null, y -> …}` for EVERY row including branch 1's own, because it never
applies a leg's own alias mapping; the per-leg routes answer `{x -> …}` for branch
1. That is a PRE-EXISTING defect of the one-shot path and the per-leg answer is the
better of the two — recorded for the lead rather than propagated.
Closed Issue #353
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…the seam pays once
Three findings from the focused delta-2 review of `2f8b1f0a`. Two of the three are
regressions my own previous remediation introduced. Nothing else is touched.
🔴 F1 — truncation and the cap-hit meter went SILENT at a leg boundary.
What the previous tree did correctly that this one still does: it never opens a
scroll for a leg the budget cannot pay for, and it never reports a cap for a
result nothing cut.
My `remaining + 1` probe is PER LEG; the budget is GLOBAL. When leg i returns
exactly `remaining` rows there is no probe row in it, and legs i+1…n are then
dropped by the `remaining <= 0` guard without anyone asking whether they had rows
— so a genuinely truncated statement reported `truncated = false`, an empty
warning and ZERO cap-hits. For a Community user (`maxQueryResults = 10000`) that
is any `UNION ALL` whose first leg holds 10,000 documents. Strictly worse than
what it replaced: `f64f475d` over-reported at one narrow boundary; this
under-reported, and the flag and the meter AGREED WITH EACH OTHER AND BOTH LIED,
so nothing could detect it. My own boundary row picked the one sub-case where the
logic is right — the budget exhausting at the end of the LAST leg — and certified
the bug.
The probe is now global as well as per-leg: reaching the skip arm means the budget
is spent and a leg is being DROPPED, which IS the truncation. It over-reports only
when every remaining leg happens to be empty — accepted as far narrower than the
corner it closes. Measured on real ES 8.18 (3-shard indices, 5 documents each),
through the licensed cap path:
legs quota rows truncated warning capHits
2 5 5 true true 1 (was false / false / 0)
3 5 5 true true 1 (was false / false / 0)
3 10 10 true true 1 (was false / false / 0)
2 11 10 false false 0
2 10 10 false false 0
3 15 15 false false 0
2 9 9 true true 1
The unit row is now the whole matrix, three-leg cases included.
F2 — the seam resolved every leg TWICE, re-EXECUTING its WHERE subquery.
What the previous tree did correctly that this one still does: it runs each leg's
phase one exactly once.
`resolveWithSchema` is not a pure check — when a leg carries a WHERE subquery it
runs `SubqueryResolver.resolve`, which EXECUTES the inner statement, and there is
no cache. Discarding the resolved statement meant every leg resolved again inside
its own `search`: measured on real ES 8.18, 2 executions of the inner query where
`f64f475d` ran 1. The guard stays — a mutation confirms it is what makes the cap
route reject a branch-type mismatch — and the fold now executes the statement the
seam resolved, so phase one is paid for once. The pin asserts the MECHANISM (the
legs handed to the fold carry no unresolved subquery) rather than only a count:
the second resolution happens inside the real `ScrollApi.scroll`, which a stub
intercepting `scroll` never reaches, so a count-only assertion was vacuous — and
measured as such before it was replaced.
F3 — my new row was vacuous and the claim it made was false.
`BudgetClient` serves `"a" -> i` for EVERY statement whatever its projection, so
no leg could produce a second key and the row-shape assertion held for any
implementation: deleting the normalisation from the fold left it GREEN. It now
uses `NamedRowsClient`, whose rows carry each leg's OWN names, and asserts what
the contract really does to a stream leg — its `amount` under `amount`, the first
branch's undeclared `category` null-filled, its own `m` following as an extra.
The `max#m` claim is DROPPED and recorded instead: where branch 1 declares a
DUPLICATE output name, `rowNormalizer` takes its legacy per-row path and a leg's
extras survive. Measured on real ES 8.18 for
`SELECT amount, amount … UNION ALL SELECT amount, MAX(amount) AS m …`:
cap route List(amount) ; List(amount, m, max#m)
per-leg route List(amount) ; List(amount, m, max#m)
one-shot List(amount) ; List(amount, max#m)
Pre-existing, not introduced by the row contract, and not fixed here — it belongs
with the by-name normalisation question in issue #354.
5 mutations re-falsified, 5 RED, MUT-1 included. core 1108 · sql 1325 ·
`++ 2.12.20 core/Test/compile`. Real ES 8.18: UnionAllCompleteness 6/6, REPL 78/78.
Closed Issue #353
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
fupelaqu
marked this pull request as ready for review
September 16, 2026 17:26
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #353
Sibling PR (arrow; merges after this one publishes a core snapshot): SOFTNETWORK-APP/softclient4es-arrow#193
Story 22.6 — set operators.
UNION,UNION ALL,UNION DISTINCT,INTERSECT [ALL]andEXCEPT [ALL]parse, validate and route;UNION ALLgains complete legs and a licensed cap thatmoves with them.
What lands
MultiSearchgainsoperators, defaulted last,with
Nilmeaning "allUNION ALL" so today's render stays byte-for-byte identical. Precedence(
INTERSECTbinds tighter) is a DERIVED tree; parentheses are not recorded.sql, at parse time: branch arity, branch types (SQLTypeUtils.matches,SELECT *exempt), noWITHon a non-first branch, and no trailingORDER BY/LIMITafter thelast branch of a non-
UNION ALLoperation.relationalClosureRequiredlearns the set-op arm; the core seamrefuses a non-
UNION ALLMultiSearchbefore any_msearch; both bridges carry a NAMED backstop;the macro aborts;
CREATE MATERIALIZED VIEWover any set operation is a parse-time 400.UNION ALLpath. One_msearchwhen every leg is bounded (byte-for-bytetoday's request). A leg that is row-shaped with NO bounding
LIMITis executed throughsearch(leg)— the SELECT without LIMIT returns only 10 rows on the non-scroll search path #209/SELECT with an explicit LIMIT above index.max_result_window fails, while the same query with NO LIMIT succeeds #224 scroll routing — and the legs are concatenated, with the licensedmaxQueryResultscap binding on the concatenated TOTAL through a running budget.SearchApi.unionAllRowNormalizer=ElasticConversion.rowNormalizerover the FIRST branch's output names, by NAME, hoisted once perstatement. The one-shot route already applied it; the per-leg route and the cap fold now do too.
Release notes — PD-7 (i)-(xi)
(i)
MultiSearcharity 2 -> 3 (operators, defaulted last) — BINARY-INCOMPATIBLE; a downstreamcase MultiSearch(a, b)must add an arm.(ii)
intersectis now a RESERVED word — a bare column or alias namedintersectmust be quoted(
"intersect"). The story's one narrowing; zero occurrences in this repo's own 1,408-statementcorpus.
(iii)
UNION [DISTINCT],INTERSECT [ALL]andEXCEPT [ALL]parse and execute at engine venues,and are refused with HTTP 400 naming
softclient4es-arrow-extensionselsewhere — a bareUNIONthat used to be a parse error is now a statement.
(iv) A trailing
ORDER BY/LIMITafter the last branch of a non-UNION ALLset operation isrejected, naming the derived-table spelling. Never parsed before, so no regression.
(v)
UNION ALLlegs that are row-shaped with NO LIMIT now return EVERY row (they returned 10 — the#209 truncation), and the licensed cap applies to the concatenated total.
(vi) 🔴 BREAKING: branch arity / type mismatches in
UNION ALLare now parse-time 400s.SELECT a, b FROM t UNION ALL SELECT a FROM u, and a BIGINT-vs-VARCHAR branch pair, used to run andreturn ragged or silently-coerced rows. The documentation promised the check as "not implemented
yet"; it is implemented now, and statements that executed before will fail.
(vii) arrow:
JoinPlanarity 5 -> 6 (setOperation);JoinPlanner.planPLANS aUNION ALLwhosebranches are cross-index JOINs where it used to refuse;
SelectStatementPatterntolerates a leading(.(viii)
SELECT * EXCEPT(cols)projection exclusion is untouched.(ix) A lone parenthesised SELECT
(SELECT … FROM …)now parses (it was a parse error); a whole setoperation wrapped in parentheses is rejected naming the unwrapped spelling.
(x) On the licensed gateway an un-LIMITed
UNION ALLcapped by quota now answersQueryRows(wasQueryStructured), and within one leg row order may differ (per-leg scroll, #238 slicing).(xi)
CREATE MATERIALIZED VIEWover ANY set operation (includingUNION ALL) is a parse-time 400where it used to throw an internal error.
(xiv) On the licensed cap path,
ElasticQuery.sqlfor a leg that carried a WHERE subquery nowcarries the RESOLVED statement (
… WHERE a IN ('v1','v2','v3')) where it carried the statement aswritten (
… WHERE a IN (SELECT b FROM y)).ScrollApi.scrollsets that field deliberately fromthe statement as written, and a pre-resolved caller defeats that. Impact is metadata and logging
only — the Elasticsearch query is built from the resolved leg either way — and the un-capped
per-leg route already behaved this way, so this is a family property rather than a new divergence.
Side effect of the AD-6 fix, worth its own line: a
LIMIT … OFFSETleg used to THROW on realElasticsearch —
action_request_validation_exception: [from] parameter must be set to 0 when [search_after] is used— because it was being scrolled. It now executes and returns correctly(
LIMIT 5 OFFSET 3+ an un-LIMITed leg over 30/24-document indices: 29 rows).Verification
sql 1325 · core 1113 · bridges 2 + 2 · macrosTests 30 ·
+ sql/compile·+ core/compile·++ 2.12.20 core/Test/compile.Real Elasticsearch 8.18:
JavaClientUnionAllCompletenessSpec6/6 (54 rows from two 3-shardindices),
JavaClient8ReplGatewayIntegrationSpec78/78. ES 6.8 / 7.17 / 9.0 left to CI.Lint:
headerCheck,scalafmtSbtCheck,scalafmtCheckandtest:scalafmtCheckall green withthe diff staged — with one caveat stated rather than hidden.
Test-scopeheaderCheckfails on46 PRE-EXISTING test files, none of them touched by this branch; CI's
headerCheckisCompile-scoped, which is why the line above passes and why the caveat is not a blocker.
Differential parser probe — the corpus, the baseline and what "narrowing" counts. Corpus: the
repo's own pre-existing statement literals, replayed through both trees; baseline pair
b7f40955(control) vs this branch. My run over 3,330 inputs recorded zero UNINTENDEDnarrowings and six widenings, all intended (five lone parenthesised SELECTs plus the bare
UNION). The independent review re-ran it over a wider extraction — 3,604 inputs, 10 verdictchanges: 7 narrowings (4 × a bare
intersectidentifier, 3 × the new branch arity/type validation)and the rest widenings. Both figures are right and they are not in conflict: every one of those 7
narrowings is a DELIBERATE, release-noted behaviour change — items (ii) and (vi) above — which my
count classified as intended rather than as a narrowing. There is no unintended verdict change in
either run. The probe was PROVED to fire: deleting
fromlessSelectfrom the control took thewidened count 6 -> 74.
Parse cost — no regression, and no timing assertion anywhere (#269/#270's lesson).
ParseCostProbe, 5 INTERLEAVED rounds: control median 3660.8 µs, branch 3563.6 µs — branch lower in5/5 rounds, −2.6 %.
ParserSpecwall time, 3 interleaved rounds: 1.834 s -> 1.686 s, lower in3/3.
Falsification: 26 mutations across three review rounds, 25 RED, restored by bytes. The one GREEN
is a redundant
UNION ALLlookahead on the arrow side, whose comment now says so.Corpus credit: zero — no captured BI statement uses a set operator.
Named residuals — tracked, NOT closed by this PR
_msearchpath never applies a leg's alias mapping, soSELECT id AS x FROM l UNION ALL SELECT id AS y FROM rreturns{x -> null, y -> …}for EVERYrow including branch 1's own. Pre-existing (measured at
b7f40955). It is why the three routesdeliberately differ on that one shape: the per-leg answer is the correct one and propagating the
one-shot defect to make them agree would ship the wrong answer on purpose.
UNION ALLroutes. (a) The licensed cap reportstruncation when the budget lands exactly on a leg boundary and every REMAINING leg is empty:
measured
SELECT a FROM t1 LIMIT 10 UNION ALL SELECT a FROM t2at quota 10 witht2empty givestruncated = trueand one cap-hit for a result nothing cut. The over-report is narrow andverified to be so —
(0,5)@5and an all-empty statement both report false — and it was adeliberate trade for the UNDER-report it replaced, which was silent. Fix direction recorded on
the issue: a
size: 0count-only search on the remaining legs turns the boundary bit from aguess into a fact without opening a scroll. (b)
unionAllByLeghands already-resolved legs tosearch, which resolves them again — the same double execution just fixed on the cap path.ScrollApi.scroll(statement, config)ignores the statement's ownLIMIT— pre-existing andlatent until this story made it reachable; worked around at the call site (only an un-LIMITed row
leg is scrolled), not fixed at the source. Recorded locally; any future caller that scrolls a
LIMITed statement inherits the silent wrong answer.
rule (3) of the cap applies per leg rather than to the aggregate;
INSERT INTO x (SELECT …) UNION (SELECT …)is indistinguishable from an INSERT column list at the scanner (handled by a control,but structurally ambiguous). All recorded locally.
🤖 Generated with Claude Code