apache / apache/paimon

[Bug] Use javaAPI to directly obtain the error result

Open
#3,679 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

0.8.1

### Compute Engine

Flink 1.17.2 and JavaAPI

### Minimal reproduce step

1. use flink sql-client to create and insert table:
```sql
CREATE CATALOG paimon_minio WITH (
'type' = 'paimon',
'warehouse' = 'file:///mnt/datadisk1/wuwenchi/tmp/warehouse'
);

CREATE TABLE `paimon_minio`.`db1`.`tb1` (
`s` string,
`ts1` TIMESTAMP(6),
`ts2` TIMESTAMP(6) WITH LOCAL TIME ZONE
) WITH (
'write-only' = 'true',
'file.format' = 'parquet'
);

insert into `paimon_minio`.`db1`.`tb1` values ('flink:2024-01-02 10:04:05.123456789',timestamp '2024-01-02 10:04:05.123456789',timestamp '2024-01-02 10:04:05.123456789');
```

2. then use JavaAPI according to https://paimon.apache.org/docs/0.8/program-api/java-api/#batch-read

```java
public void read() throws TableNotExistException, IOException {
Map props = new HashMap<>();
props.put("warehouse", "file:///mnt/datadisk1/wuwenchi/tmp/warehouse");

CatalogContext ctx = CatalogContext.create(new Options(props));
Catalog catalog = CatalogFactory.createCatalog(ctx);
Table table = catalog.getTable(Identifier.create("db1", "tb1"));

ReadBuilder rb = table.newReadBuilder();
List splits1 = rb.newScan().plan().splits();
TableRead read = rb.newRead();
RecordReader reader = read.createReader(splits1);
reader.forEachRemaining(record -> {
System.out.println(record.getString(0));

Timestamp ts1 = record.getTimestamp(1, 6);
System.out.println(ts1);

Timestamp ts2 = record.getTimestamp(2, 6);
System.out.println(ts2);
});
}
```

print in console:
```
flink:2024-01-02 10:04:05.123456789
2024-01-02T10:04:05.123456
2024-01-02T02:04:05.123456
```

It is right for `s` and `ts1`, but it is wrong for `ts2`, we expect `2024-01-02T10:04:05.123456` for `ts2`.
Is this 8 hours missing a time zone?

### What doesn't meet your expectations?

Use JavaAPI to directly obtain the correct result.

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

The reproduction uses CatalogFactory.createCatalog, Table.newReadBuilder(), TableRead.createReader(), and InternalRow.getTimestamp(2, 6); start by tracing how TIMESTAMP WITH LOCAL TIME ZONE is decoded through these Java API entry points. Compare the timezone handling with the expected ts2 value and verify the corrected result with the supplied Flink/JavaAPI reproduction.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.