cockroachdb / cockroachdb/cockroach
Decimal to Decimal Cast Can Store Wrong Scale
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
DECIMALS in CockroachDB and Postgres have a precision and a scale component. The precision is the total number of base 10 digits in the result and the scale is the number of digits to the right of the decimal point. The precision and scale are optional parameters and when defining a DECIMAL type they may be specified as DECIMAL(precision, scale). For example the TPCC schema uses DECIMAL(5, 2) to represent prices between -100.00 and 100.00.
Internally, Cockroach represents decimals as an integer*10^exponent where `exponent` is the '-scale'. So the price 12.00 stored in the type DECIMAL(5,2) is represented as 1200 * 10 ^ -2.
There is a bug in the [Decimal to Decimal(precision, scale) type cast](https://github.com/cockroachdb/cockroach/blob/a1da8ae85971cc2deda34b80d9a04f96739b28b4/pkg/sql/sem/tree/datum.go#L6482). It exits early if scale == exponent. This code path is trying to identify the case where the decimal is already in the correct format, but it should really be checking scale == -exponent since the scale and exponent are expected to be equal, but with opposite signs.
This was discovered as part of LDR testing. We fingerprint the source and the destination clusters to ensure replication copied everything correctly. As part of replication, we were fixing the representation of the decimals which caused the destination fingerprint to differ from the source fingerprint.
Jira issue: CRDB-49391
Contributor guide
Assessment
This issue has not been assessed yet.