cockroachdb / cockroachdb/cockroach
sql: SQRDIFF function returns negative value for large DECIMAL when all values are equal, with magnitude varying by execution plan
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Describe the problem**
When all values of a DECIMAL column `c1` are the same huge number (`99999999999999999999.9999999999`), `SQRDIFF(c1)` should return `0` (the sum of squared differences from the mean is zero when all values are identical).
Instead, CockroachDB v26.2.3 returns a large **negative** value, and the magnitude depends on the query execution plan:
- Single‑table scan (local plan): **-20000000000**
- Lateral JOIN query producing the same rows (full‑distribution plan with merge join): **-30000000000**
**To Reproduce**
**What did you do?**
Ran `SQRDIFF(c1)` on a DECIMAL column where every value is the same huge number, once on a single table and once through a lateral JOIN that forces a different aggregation plan.
**Steps to reproduce:**
1. Start a three‑node CockroachDB v26.2.3 cluster (insecure mode).
2. Connect with psql.
3. Run the following SQL:
```sql
DROP DATABASE IF EXISTS repro_cockroachdb803_db6_sqrdiff_plan CASCADE;
CREATE DATABASE repro_cockroachdb803_db6_sqrdiff_plan;
USE repro_cockroachdb803_db6_sqrdiff_plan;
SET vectorize = 'off';
SET distsql = 'always';
CREATE TABLE source (
vp_rowid INT8 PRIMARY KEY,
c1 DECIMAL NOT NULL
);
CREATE TABLE vp_l (
vp_rowid INT8 PRIMARY KEY,
c1 DECIMAL NOT NULL
);
CREATE TABLE vp_r (
vp_rowid INT8 PRIMARY KEY
);
INSERT INTO source
SELECT i, 99999999999999999999.9999999999::DECIMAL
FROM generate_series(1, 11) AS g(i);
INSERT INTO vp_l SELECT vp_rowid, c1 FROM source;
INSERT INTO vp_r SELECT vp_rowid FROM source;
ALTER TABLE source SPLIT AT VALUES (3), (5), (7), (9);
ALTER TABLE vp_l SPLIT AT VALUES (3), (5), (7), (9);
ALTER TABLE vp_r SPLIT AT VALUES (3), (5), (7), (9);
ALTER TABLE source SCATTER;
ALTER TABLE vp_l SCATTER;
ALTER TABLE vp_r SCATTER;
-- Single‑table query
SELECT 'single' AS shape, SQRDIFF(c1)
FROM source;
-- Lateral JOIN query with the same row set
SELECT 'lateral' AS shape, SQRDIFF(c1)
FROM vp_l l
JOIN LATERAL (
SELECT *
FROM vp_r r
WHERE r.vp_rowid = l.vp_rowid
) r ON TRUE;
```
4. Observe the results:
```
shape | sqrdiff
---------+-------------
single | -20000000000
lateral | -30000000000
```
Both are negative; the correct `SQRDIFF` for this data set is `0` (all values equal).
The two plans produce different negative magnitudes, demonstrating that the error is sensitive to how partial aggregations are formed.
**Expected behavior**
`SQRDIFF(c1)` must return `0` when all input values are identical. A tiny non‑negative rounding error is tolerable, but a large negative value that depends on the query plan is a bug.
**Additional data / screenshots**
Execution plans:
**Single‑table (`'single'`):**
```
scalar-group-by
scan source
aggregations
sqr-diff(c1)
```
**Lateral JOIN (`'lateral'`):**
```
scalar-group-by
inner-join (merge)
scan vp_r
scan vp_l
aggregations
sqr-diff(c1)
```
The difference arises from how rows are partitioned for partial aggregation before the final merge. In the single‑table scan, aggregation happens directly on source ranges; in the lateral plan, a merge join brings together vp_l and vp_r, which likely changes the grouping boundaries for the local aggregation stages, leading to a different truncation error.
**Environment:**
- **CockroachDB version:** CockroachDB CCL v26.2.3
- **Docker image:** cockroachdb/cockroach:v26.2.3
- **Server OS:** Ubuntu 20.04.6 LTS x86_64, Linux 5.4.0-204-generic
- **Client app:** psql (PostgreSQL connection, sslmode=disable)
- **Cluster mode:** three‑node, insecure. The test set `vectorize = 'off'` and `distsql = 'always'`, and executed `SPLIT AT` and `SCATTER` on `source`, `vp_l`, and `vp_r` to create multiple ranges with multi‑node placement. The single‑table scan and the semantically equivalent lateral join reconstruction produced different `SQRDIFF` results.
**Additional context**
This is a precision and serialisation defect in the row‑oriented engine for `SQRDIFF` on DECIMAL values. The formula `sum(c1²) - (sum(c1))²/count` involves extremely large intermediate values, and the serialisation of partial aggregates loses low‑order digits, preventing exact cancellation. The magnitude of the error changes with the execution plan because different plans partition the data differently for local aggregation.
Jira issue: CRDB-66366
Contributor guide
Assessment
This issue has not been assessed yet.