Skip to content

A second corpus: the statements a real Tableau emitted, replayable without Tableau - #329

Merged
fupelaqu merged 1 commit into
mainfrom
feature/tableau-live-corpus
Sep 14, 2026
Merged

fupelaqu merged 1 commit into
mainfrom
feature/tableau-live-corpus

Conversation

@fupelaqu

Copy link
Copy Markdown
Contributor

A second corpus: what a real Tableau emitted, replayable without Tableau

Until now, "does Tableau work?" could only be answered by installing Tableau. This adds the statements a
real Tableau Desktop 2026 for macOS actually sent, as a tracked test resource with its own replay
suite, so a clean checkout can tell you when Tableau support regresses.

Captured 2026-09-13 over the JDBC driver 0.3.3-SNAPSHOT (core 0.23.0-SNAPSHOT) against real
Elasticsearch 8.18.3, with p6spy at the JDBC boundary. 127 statements across six sessions → 26
distinct shapes.
Normalisation is applied to the grouping key only; every committed statement is
verbatim.

This is a SECOND corpus, deliberately

epic-19-bi-corpus.csv is untouched, and so are epic-21-attribution.csv, baseline-pre-epic21.csv
and CorpusReplaySpec (byte-identical; CorpusReplaySpec stays 12/12 with its summary line
unchanged). That corpus is a fixed baseline: the published 56/99 and 56/75 and the Epic 23 series
are anchored to it, and appending would silently move denominators the epic already published. The new
file has no baseline, no series and no denominators.

What the rows are

owner rows meaning
works 14 preview, aggregate, filter, sort, SELECT 1, SELECT * … LIMIT 1
issue:328 5 the HAVING (COUNT(1) > 0) family — must parse; this is the regression guard for the fix that unblocked Tableau's data-source preview
epic22a_derived_table 3 the connect-time SELECT … FROM (SELECT 1 AS \COL`) AS `SUBQUERY`` — rejected, Epic 22 owns it
capability_open 2 the temp-table probe pair
rejected_by_design 2 GROUP BY 2 against a one-item SELECT list — an out-of-range ordinal, correctly rejected

21 of the 26 parse; 0 throw; 0 come back as an internal parser error. 14 shapes are absent from the
Epic 19 corpus.

Three things the capture corrected:

  • The MySQL dialect's temp-table probe is a plain CREATE TABLE, not CREATE TEMPORARY TABLE. It
    parses and is refused at execution by the index-name validator. The parse-time refusal added in Tableau's temp-table probe is refused at the wrong layer, for a misleading reason, and one line from creating the indices for real #326
    covers the SQL-92 TEMPORARY spelling, which this dialect never emits. The suite guards the label and
    the verdict; refusing these would be a product decision, not a bug fix.
  • The connect-time derived table is the most frequent shape in the whole capture — 17 of 127, once
    per connection.
  • GROUP BY 2 against a one-item SELECT list is Tableau's ordinal capability probe's negative
    control
    , issued immediately after GROUP BY 1. Our rejection, naming the valid range, is the
    correct answer.

Gate integrity

Four id sets and the parse total are pinned in compiled code, asserted two-way against the CSV.
Falsified with 9 mutations, 9 predicted REDs, restored by captured bytes. The one that justifies
pinning the total: a statement that regresses while its expected cell is edited to absorb the
change
passes the verdict gate — only the pinned count catches it.

The suite reuses CapturedSqlProbe.parseCsv, CorpusReplay.replayAll and CorpusReplay.Outcome. No
second codec, no second replay loop.

What this does NOT tell you

It is parse-only by construction — the sql module has no Elasticsearch client — so it proves these
shapes are accepted, never that they answer correctly. The README names which rows already have
oracles in GroupByCompletenessSpec and which have none; four works rows have none at all. That is a
clean pickup for a later story and the honest next step for any claim about Tableau.

It also records what Tableau emitted for those interactions on that version. It is not a substitute
for running Tableau, and the README says so.

⚠️ Recorded and deliberately not explained: this capture has Tableau emitting GROUP BY 1 where the
August corpus has GROUP BY \t`.`category`` for the same interaction. A control run rules out the
Tableau customization file; the cause is unestablished, so both spellings are recorded and neither is
given a story.

Verification

TableauLiveReplaySpec 11/11 · CorpusReplaySpec 12/12 · sql/test 1104 · + compile · the CI lint
line green · scalafmtAll with files staged first. Tests on 2.13 only, as CI runs them. No ES legs —
parse-only by construction.

Content scan over all 26 statements: zero hits for host, credential or user patterns. The only
identifiers present are the synthetic fixture's tables and columns, Tableau's own generated names, three
country literals, and the test cluster name.

🤖 Generated with Claude Code

…able without Tableau

A live Tableau Desktop 2026 for macOS was driven against real Elasticsearch 8.18.3
through the JDBC driver on 2026-09-13 and captured with p6spy. Those 127 statements
existed only as log files on one machine, so Tableau support could not be
regression-tested without Tableau. This commit closes that gap.

New, beside the Epic 19 corpus and deliberately NOT merged into it (that corpus is a
fixed baseline the published N/99 and N/75 figures are anchored to, and appending to it
would silently move a published denominator):

  sql/src/test/resources/corpus/tableau-live-2026-09-13.csv
  sql/src/test/scala/.../census/TableauLiveReplaySpec.scala

26 rows, one per distinct normalised shape, each holding one verbatim representative and
its occurrence count; the counts sum to the full 127, which the suite asserts. Only two
Tableau-generated name families are normalised, and only for the grouping key. The suite
reuses CorpusReplay's codec, replay loop and Outcome - no second CSV reader, no second
replay loop - and CorpusReplaySpec is untouched at 12/12.

Measured, not predicted: 21 of the 26 shapes parse. Three results correct an assumption:

  - the temp-table probe is NOT refused. Tableau's MySQL dialect emits a plain
    CREATE TABLE / DROP TABLE, which issue #326's CREATE [LOCAL | GLOBAL] TEMPORARY
    TABLE refusal never reaches. They parse, exactly as the Epic 19 twins do, and the
    open product decision is what the suite guards.
  - GROUP BY 2 against a one-item SELECT list is Tableau's ordinal capability probe,
    negative control. Rejecting it is the correct answer (#298 names the valid range).
  - the connect-time derived table is the most frequent shape in the whole capture
    (17 of 127, once per connection). Epic 22 owns it; it does not block browsing.

Four id sets and the parse total are PINNED IN COMPILED CODE with two-way set equality
against the CSV, so no cell edit can absorb a regression: the issue #328 row-existence
check must keep parsing, the probes must stay labelled capability_open, the derived
tables must stay owned by Epic 22. Falsified with 9 mutations, 9 predicted REDs -
including the composite one that matters, a statement that regresses while its expected
cell is flipped to hide it, which only the pinned parse total catches.

Recorded and deliberately not explained: this capture has Tableau emitting GROUP BY 1
where the August capture recorded GROUP BY `t`.`category` for the same interaction, and
a control run showed the difference is not caused by a .tdc. Both shapes are kept.

Correctness is out of scope here - the sql module has no Elasticsearch client, so every
gate is a parse verdict. The README names which rows already have an oracle in
GroupByCompletenessSpec and which would need a new one.

Every statement was read before it was committed; a scan for host-, user- and
secret-shaped content returns zero hits. The file inherits the corpus -text rule, whose
comment now says why it applies here too (the statements carry literal TABs).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@fupelaqu
fupelaqu marked this pull request as ready for review September 14, 2026 09:11
@fupelaqu
fupelaqu merged commit 40c8c63 into main Sep 14, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant