pingcap / pingcap/tiflash

[Question] Decimal `Field` Comparison Conflict in C++ and SQL Logic

Open
#6,065 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type/question
Dominant language
C++
Stars
1k
Forks
423
Avg merge
1d 15h
Merged PRs (30d)
24

Description

General Question

I notice the equality comparison of Decimal Field is different in TiFlash C++ Codebase and SQL logic.

TiFlash C++ Codebase

First, we know that there are four different subtype of Decimal: https://github.com/pingcap/tiflash/blob/c8624a048783d6db172b000a9ec180848f6fb1c4/dbms/src/Core/Field.h#L224-L227

== Operator Overloading

If we want to compare two Fields, we will see the == operator is already overloaded. But this operator first compares the types of two Fields: https://github.com/pingcap/tiflash/blob/c8624a048783d6db172b000a9ec180848f6fb1c4/dbms/src/Core/Field.h#L510-L513

Thus if we have one Decimal32 and one Decimal64, then they are always not equal due to different typing. Such a normal C++ == operator is called as following code segment:

Field col1_field;
col_1.column->get(i, col1_field);
Field col2_field;
col_2.column->get(i, col2_field);
const bool equals = (col1_field == col2_field);

SQL Logic

However, in the SQL we can define a DECIMAL datatype with specifying its precision and scale. Additionly, we can even compare the the value equality of DECIMALs in different precisions and scales.

You can find such a comparison in the picture below:

test

The Comparison

We see the underlying values inside each DECIMAL are correctly compared. The logic of comparison is much more complex than the == operator above.
This SQL Level Comparison is implemented by the Function provided by the Clickhouse:

My Question

  • Do we have any correct way to compare two Decimal Fields?
  • Is this conflict true what we want in flash?

How I come to this Question

I am write unit tests of a comparison. So I have implemented with Function and try to use Field comparison to validate my implementation. My code is fine with other data types, but have issue only with Decimal.

TODO

I will check later how the Clickhouse handle this Field issue.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Decimal subtype and Field equality definitions in dbms/src/Core/Field.h, then trace the SQL comparison entry point in dbms/src/Functions/FunctionsComparison.h and dbms/src/Core/DecimalComparison.h. Compare those semantics with the unit-test use case described in the issue; done means establishing the intended Decimal Field comparison behavior and documenting or testing the correct approach.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.