A second corpus: the statements a real Tableau emitted, replayable without Tableau - #329
Merged
Merged
Conversation
…able without Tableau
A live Tableau Desktop 2026 for macOS was driven against real Elasticsearch 8.18.3
through the JDBC driver on 2026-09-13 and captured with p6spy. Those 127 statements
existed only as log files on one machine, so Tableau support could not be
regression-tested without Tableau. This commit closes that gap.
New, beside the Epic 19 corpus and deliberately NOT merged into it (that corpus is a
fixed baseline the published N/99 and N/75 figures are anchored to, and appending to it
would silently move a published denominator):
sql/src/test/resources/corpus/tableau-live-2026-09-13.csv
sql/src/test/scala/.../census/TableauLiveReplaySpec.scala
26 rows, one per distinct normalised shape, each holding one verbatim representative and
its occurrence count; the counts sum to the full 127, which the suite asserts. Only two
Tableau-generated name families are normalised, and only for the grouping key. The suite
reuses CorpusReplay's codec, replay loop and Outcome - no second CSV reader, no second
replay loop - and CorpusReplaySpec is untouched at 12/12.
Measured, not predicted: 21 of the 26 shapes parse. Three results correct an assumption:
- the temp-table probe is NOT refused. Tableau's MySQL dialect emits a plain
CREATE TABLE / DROP TABLE, which issue #326's CREATE [LOCAL | GLOBAL] TEMPORARY
TABLE refusal never reaches. They parse, exactly as the Epic 19 twins do, and the
open product decision is what the suite guards.
- GROUP BY 2 against a one-item SELECT list is Tableau's ordinal capability probe,
negative control. Rejecting it is the correct answer (#298 names the valid range).
- the connect-time derived table is the most frequent shape in the whole capture
(17 of 127, once per connection). Epic 22 owns it; it does not block browsing.
Four id sets and the parse total are PINNED IN COMPILED CODE with two-way set equality
against the CSV, so no cell edit can absorb a regression: the issue #328 row-existence
check must keep parsing, the probes must stay labelled capability_open, the derived
tables must stay owned by Epic 22. Falsified with 9 mutations, 9 predicted REDs -
including the composite one that matters, a statement that regresses while its expected
cell is flipped to hide it, which only the pinned parse total catches.
Recorded and deliberately not explained: this capture has Tableau emitting GROUP BY 1
where the August capture recorded GROUP BY `t`.`category` for the same interaction, and
a control run showed the difference is not caused by a .tdc. Both shapes are kept.
Correctness is out of scope here - the sql module has no Elasticsearch client, so every
gate is a parse verdict. The README names which rows already have an oracle in
GroupByCompletenessSpec and which would need a new one.
Every statement was read before it was committed; a scan for host-, user- and
secret-shaped content returns zero hits. The file inherits the corpus -text rule, whose
comment now says why it applies here too (the statements carry literal TABs).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A second corpus: what a real Tableau emitted, replayable without Tableau
Until now, "does Tableau work?" could only be answered by installing Tableau. This adds the statements a
real Tableau Desktop 2026 for macOS actually sent, as a tracked test resource with its own replay
suite, so a clean checkout can tell you when Tableau support regresses.
Captured 2026-09-13 over the JDBC driver
0.3.3-SNAPSHOT(core0.23.0-SNAPSHOT) against realElasticsearch 8.18.3, with p6spy at the JDBC boundary. 127 statements across six sessions → 26
distinct shapes. Normalisation is applied to the grouping key only; every committed statement is
verbatim.
This is a SECOND corpus, deliberately
epic-19-bi-corpus.csvis untouched, and so areepic-21-attribution.csv,baseline-pre-epic21.csvand
CorpusReplaySpec(byte-identical;CorpusReplaySpecstays 12/12 with its summary lineunchanged). That corpus is a fixed baseline: the published
56/99and56/75and the Epic 23 seriesare anchored to it, and appending would silently move denominators the epic already published. The new
file has no baseline, no series and no denominators.
What the rows are
worksSELECT 1,SELECT * … LIMIT 1issue:328HAVING (COUNT(1) > 0)family — must parse; this is the regression guard for the fix that unblocked Tableau's data-source previewepic22a_derived_tableSELECT … FROM (SELECT 1 AS \COL`) AS `SUBQUERY`` — rejected, Epic 22 owns itcapability_openrejected_by_designGROUP BY 2against a one-item SELECT list — an out-of-range ordinal, correctly rejected21 of the 26 parse; 0 throw; 0 come back as an internal parser error. 14 shapes are absent from the
Epic 19 corpus.
Three things the capture corrected:
CREATE TABLE, notCREATE TEMPORARY TABLE. Itparses and is refused at execution by the index-name validator. The parse-time refusal added in Tableau's temp-table probe is refused at the wrong layer, for a misleading reason, and one line from creating the indices for real #326
covers the SQL-92
TEMPORARYspelling, which this dialect never emits. The suite guards the label andthe verdict; refusing these would be a product decision, not a bug fix.
per connection.
GROUP BY 2against a one-item SELECT list is Tableau's ordinal capability probe's negativecontrol, issued immediately after
GROUP BY 1. Our rejection, naming the valid range, is thecorrect answer.
Gate integrity
Four id sets and the parse total are pinned in compiled code, asserted two-way against the CSV.
Falsified with 9 mutations, 9 predicted REDs, restored by captured bytes. The one that justifies
pinning the total: a statement that regresses while its
expectedcell is edited to absorb thechange passes the verdict gate — only the pinned count catches it.
The suite reuses
CapturedSqlProbe.parseCsv,CorpusReplay.replayAllandCorpusReplay.Outcome. Nosecond codec, no second replay loop.
What this does NOT tell you
It is parse-only by construction — the
sqlmodule has no Elasticsearch client — so it proves theseshapes are accepted, never that they answer correctly. The README names which rows already have
oracles in
GroupByCompletenessSpecand which have none; fourworksrows have none at all. That is aclean pickup for a later story and the honest next step for any claim about Tableau.
It also records what Tableau emitted for those interactions on that version. It is not a substitute
for running Tableau, and the README says so.
GROUP BY 1where theAugust corpus has
GROUP BY \t`.`category`` for the same interaction. A control run rules out theTableau customization file; the cause is unestablished, so both spellings are recorded and neither is
given a story.
Verification
TableauLiveReplaySpec11/11 ·CorpusReplaySpec12/12 ·sql/test1104 ·+ compile· the CI lintline green ·
scalafmtAllwith files staged first. Tests on 2.13 only, as CI runs them. No ES legs —parse-only by construction.
Content scan over all 26 statements: zero hits for host, credential or user patterns. The only
identifiers present are the synthetic fixture's tables and columns, Tableau's own generated names, three
country literals, and the test cluster name.
🤖 Generated with Claude Code