apache / apache/gluten

[VL] Incorrect result type of nested decimal arithmetic expressions

Open
#7,082 8 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

The SQL as follows will produce wrong result:
```SQL
select col0 / (col1 + 0.00001) from table;
```
where col0 and col1 is decimal(20, 0).
The values of col0 and col1 are 25 and 72. Gluten returns 3.4722217399692027821 and spark returns 0.3472221739969202782.
The result type needs to be `decimal(38, 19)`. But during calculation, the actual result type is `decimal(38, 20)`:
image
The reason is that the type of the subexpression `col1 + 0.00001` will be rescaled to `decimal(27, 5)`:
截屏2024-09-02 17 15 29
But we don't notice it when we rescale the topmost expression:
截屏2024-09-02 17 16 06

### Spark version

None

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

_No response_

Contributor guide

Open the contributing guide

Research direction

Reproduce the SQL query `select col0 / (col1 + 0.00001) from table` with decimal(20, 0) operands in the Velox backend and inspect the inferred types of the nested and top-level expressions. Compare Gluten's result and decimal(38, 20) type with Spark's expected result and decimal(38, 19) type; done means nested decimal arithmetic produces the correct result type and value.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.