Skip to content

A decimal longer than a double keeps every digit, so 12345678901234567.89 is not 1.2345678901234568e16 - #1413

Merged
WaylandYang merged 4 commits into
deeplethe:devfrom
rudycelekli:coerce-exact-decimal
Oct 27, 2026
Merged

WaylandYang merged 4 commits into
deeplethe:devfrom
rudycelekli:coerce-exact-decimal

Conversation

@rudycelekli

@rudycelekli rudycelekli commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Why

coerce turns a numeric string into a JSON number. When the value is not an i64, it parses it as f64, whatever the column type. An f64 holds about 15 significant digits. An exact DECIMAL, NUMERIC, NUMBER or Snowflake fixed value with more digits than that is silently rounded before it reaches the model.

From a live MariaDB 11.4, a DECIMAL(20,2) holding 12345678901234567.89 came back as 1.2345678901234568e+16. That is a different amount, and the model quotes it as the stored figure. The same happens to Snowflake NUMBER(38,0) identifiers past i64, and to Databricks DECIMAL columns, because both go through coerce.

The codebase already intends to keep these exact. The comment on MySQL's Cell::Decimal says that values beyond f64 precision stay strings. ClickHouse's cell keeps a Decimal(P, S) with P > 15 as a string for the same reason. The shared coerce did not follow either rule.

What changes

For an exact numeric type, coerce only converts through f64 when the value has at most 15 significant digits. Leading and trailing zeros are ignored, so 1234.50 is still 1234.5 and 100000000000000000000 is still 1e20. A longer value stays as the string the database sent. DOUBLE, FLOAT and REAL are unchanged, because an approximate type has nothing more to lose.

How it was checked

  • A new unit test, an_exact_decimal_longer_than_a_double_stays_exact, covers DECIMAL(20,2) and fixed past 15 digits. It checks that a short decimal, a trailing-zero integer and a DOUBLE are still numbers.
  • A new live test, a_decimal_longer_than_a_double_keeps_every_digit_mysql, uses the Fx fixture. It expects {"total":"12345678901234567.89","fee":1234.5}.
  • UTOPIA_TEST_MYSQL_URL=mysql://root:pw@127.0.0.1:13306/sales cargo test -p utopia-server --bins query_engine, run against mariadb:11.4: 74 passed.
  • With the fix reverted, the unit test fails with left: Number(1.2345678901234568e+16), right: String("12345678901234567.89"). The live test fails with left: ["{\"total\":1.2345678901234568e+16,\"fee\":1234.5}"].
  • cargo fmt --all --check and cargo clippy -p utopia-server --all-targets -- -D warnings pass.

Before review

  • Every commit is signed off (git commit -s)
  • cargo fmt --all --check, cargo clippy and cargo test pass for the touched crate

Requested ADR follow-up

Updated ADR 0018 as requested in review. This appended revision changes documentation only; Rust and Cargo files are byte-for-byte unchanged from the prior public head. cargo fmt --all --check and git diff --check pass. No new Rust test run or live-cluster acceptance is claimed for this revision. Final-head CI remains required.

…7.89 is not 1.2345678901234568e16

Signed-off-by: Rudy Celekli <rudy@gradiahq.com>

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only affects values over 15 significant digits, which is the intended behaviour. Please update the type-restoration open question in ADR 0018 to say that long exact values stay strings. Optionally, keep the number when the f64 prints back to the same digits.

rudycelekli and others added 2 commits October 27, 2026 00:15
Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. This conflicted with #1410 only because both added a test at the same spot in mysql.rs; I merged dev into the branch and kept both tests. Approving and merging.

@WaylandYang
WaylandYang merged commit 8630da5 into deeplethe:dev Oct 27, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants