apache / apache/gluten

[CH] Support more decimal expressions in Spark compatible way

Open
#1,150 0 comments 0 reactions 0 assignees View on GitHub
enhancement stale
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 21h
Merged PRs (30d)
85

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Now gluten with ch backend does not support decimal expressions like UnscaledValue and MakeDecimal.
Also, there are compatible issues in current implementation of PromotePrecision and CheckOverflow.
ClickHouse already suppport decimal data type, but is not compatible with spark in many ways.For example, arithmetic rules and error handlings are quite different.
We need to deal with these situation carefully.

**Describe the solution you'd like**
First , UnscaledValue and MakeDecimal will be supported.
Second, a compute conversion rule will be add .

More work will be discovered later

Contributor guide

Open the contributing guide

Research direction

Start by locating the existing implementations of UnscaledValue, MakeDecimal, PromotePrecision, and CheckOverflow, then compare their behavior with Spark and ClickHouse decimal rules. Add support for the two missing expressions and the compute conversion rule, with completion confirmed by compatible arithmetic and error-handling behavior; further decimal work may remain.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, scala, spark
Domain
backend, data, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.