[Bug] Decimal Value Corruption in Batch Write (Values Shifted Incorrectly)
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
From paimon-bundle version 0.9.0 to 1.1.1
### Compute Engine
flink 1.19.1
### Minimal reproduce step
Description:
I'm using paimon-bundle for Batch Write operations and encountering incorrect decimal field values after writing. The issue persists across versions from 0.9.0 to 1.1.1.
Observed Behavior:
1、When writing a 2-decimal-place value (e.g., 1.23), the result gets left-shifted by 2 places (becomes 0.0123)
2、When writing a 3-decimal-place value, the result gets left-shifted by 1 place
3、4-decimal-place values appear correct
Reproduction Code:
SQL:
```
CREATE CATALOG xxxx ....'type' = 'paimon'....;
use xxxx;
CREATE TABLE `tmp`.`test` (
`aa` BIGINT NOT NULL ,
`bb` TIMESTAMP ( 3 ),
`aaa` STRING ,
`bbb` DECIMAL ( 12, 4 ) ,
CONSTRAINT `PK_aa` PRIMARY KEY ( `aa` ) NOT ENFORCED
) WITH ( )
```
JAVA:
```
Identifier identifier = Identifier.create(dbName, tableName);
org.apache.paimon.table.Table table = catalog.getTable(identifier);
BatchWriteBuilder writeBuilder = table.newBatchWriteBuilder();
BatchTableWrite write = writeBuilder.newWrite();
// First record
Long r1_col1 = Long.valueOf(111);
BinaryString r1_col2 = BinaryString.fromString("aaa");
BigDecimal bd = new BigDecimal("1.23");
Decimal r1_col3 = org.apache.paimon.data.Decimal.fromBigDecimal(bd, bd.precision(), bd.scale());
System.out.println(r1_col3);
GenericRow record1 = GenericRow.of(r1_col1, null, r1_col2, r1_col3);
write.write(record1, 1); // Using fixed bucket number 1
// Second record
Long r2_col1 = Long.valueOf(222);
BinaryString r2_col2 = BinaryString.fromString("bbb");
BigDecimal bd2 = new BigDecimal("333333.3333");
Decimal r2_col3 = org.apache.paimon.data.Decimal.fromBigDecimal(bd2, bd2.precision(), bd2.scale());
GenericRow record2 = GenericRow.of(r2_col1, null, r2_col2, r2_col3);
write.write(record2, 1);
List messages = write.prepareCommit();
BatchTableCommit commit = writeBuilder.newCommit();
commit.commit(messages);
System.out.println("done");
```
Actual Output:
0.0123
333333.3333
### What doesn't meet your expectations?
The correct query output should be:
1.23
333333.3333
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running the supplied SQL and Java reproduction with paimon-bundle 0.9.0–1.1.1, focusing on Decimal.fromBigDecimal and the BatchWriteBuilder/BatchTableWrite path. Compare the stored query results for 1.23 and 333333.3333 with the expected values; done means decimal values retain their declared scale during batch writes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100