[Bug] timestamp reading from paimon is different from hive after hive-cloning
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
Paimon's version: 1.2-snapshot(2025/06/09)
### Compute Engine
Hive's version: hive 2.3.10
Flink's version: 1.20.1
### Minimal reproduce step
When selecting the paimon table which was cloned from hive through hive-clone action, the selected result for timestamp is different between paimon and hive.
**Note: file format is parquet.**
Minimal reproduce step:
1. create a table in hive and insert some data:
```sql
-- create a table
CREATE TABLE test.hive_test_table(
`a` int COMMENT 'The a field',
`ts` timestamp COMMENT 'The ts field'
) STORED AS PARQUET;
-- insert a value
INSERT INTO test.hive_test_table VALUES (1, '2025-06-03 16:00:00');
-- select
SELECT * FROM test.hive_test_table;
-- result: 1, 2025-06-03T16:00
```
2. run hive-clone action
3. select the cloned paimon table in flink
```sql
SELECT * FROM paimon_hive_test_table;
-- result: 1, 2025-06-03T08:00
```
The reaseon may be:
Hive will transform the timestamp to UTC for storage, and transform the stored timestamp to `writer.time.zone` or system default time zone when reading. While paimon will directly read the original timestamp stored in parquet file without any transforms, this leads to difference.
### What doesn't meet your expectations?
The selected result for timestamp has changed after cloning to paimon.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Run the listed Hive SQL setup, hive-clone action, and Flink query using the stated versions and Parquet format. Compare the timestamp before and after cloning, then trace the clone and timestamp-reading paths to determine where timezone handling diverges. Done means the cloned Paimon query returns the same timestamp as Hive.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100