[Question] Decimal `Field` Comparison Conflict in C++ and SQL Logic
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1k
- Forks
- 423
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 24
Description
General Question
I notice the equality comparison of Decimal Field is different in TiFlash C++ Codebase and SQL logic.
TiFlash C++ Codebase
First, we know that there are four different subtype of Decimal: https://github.com/pingcap/tiflash/blob/c8624a048783d6db172b000a9ec180848f6fb1c4/dbms/src/Core/Field.h#L224-L227
== Operator Overloading
If we want to compare two Fields, we will see the == operator is already overloaded. But this operator first compares the types of two Fields: https://github.com/pingcap/tiflash/blob/c8624a048783d6db172b000a9ec180848f6fb1c4/dbms/src/Core/Field.h#L510-L513
Thus if we have one Decimal32 and one Decimal64, then they are always not equal due to different typing. Such a normal C++ == operator is called as following code segment:
Field col1_field;
col_1.column->get(i, col1_field);
Field col2_field;
col_2.column->get(i, col2_field);
const bool equals = (col1_field == col2_field);
SQL Logic
However, in the SQL we can define a DECIMAL datatype with specifying its precision and scale. Additionly, we can even compare the the value equality of DECIMALs in different precisions and scales.
You can find such a comparison in the picture below:

The Comparison
We see the underlying values inside each DECIMAL are correctly compared. The logic of comparison is much more complex than the == operator above.
This SQL Level Comparison is implemented by the Function provided by the Clickhouse:
- https://github.com/pingcap/tiflash/blob/dfac6a50b09711d12a3b6909126782d1b8ed363d/dbms/src/Functions/FunctionsComparison.h#L1481
- https://github.com/pingcap/tiflash/blob/eaf1a4cf886d1d5a86d04ef14eff956778ab4911/dbms/src/Core/DecimalComparison.h#L66
My Question
- Do we have any correct way to compare two Decimal
Fields? - Is this conflict true what we want in flash?
How I come to this Question
I am write unit tests of a comparison. So I have implemented with Function and try to use Field comparison to validate my implementation. My code is fine with other data types, but have issue only with Decimal.
TODO
I will check later how the Clickhouse handle this Field issue.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Decimal subtype and Field equality definitions in dbms/src/Core/Field.h, then trace the SQL comparison entry point in dbms/src/Functions/FunctionsComparison.h and dbms/src/Core/DecimalComparison.h. Compare those semantics with the unit-test use case described in the issue; done means establishing the intended Decimal Field comparison behavior and documenting or testing the correct approach.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100