apache / apache/fluss

[common] Negative timestamps fail to decode in Arrow and Iceberg

Open
#4,295 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Fluss version

main (development)

### Please describe the bug 🐞

Valid pre-epoch timestamps with a non-zero sub-millisecond component throw `IllegalArgumentException` when decoded through:
- Arrow readers for `TIMESTAMP` and `TIMESTAMP_LTZ` at microsecond/nanosecond precision.
- `IcebergKeyDecoder` for timestamp keys.

For example, round-tripping `1969-12-31 23:59:59.999999` (-1µs) fails, while -1000µs succeeds.

The decoders use `/` and `%`, producing a negative nano-of-millisecond remainder that violates the timestamp constructor's invariant. Valid negative timestamps should round-trip unchanged.

### Solution

Reuse the existing microsecond timestamp factories and use `Math.floorDiv` / `Math.floorMod` for Arrow nanoseconds.

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Arrow readers for TIMESTAMP and TIMESTAMP_LTZ at microsecond and nanosecond precision, then inspect IcebergKeyDecoder's timestamp-key path. Reproduce the failure with 1969-12-31 23:59:59.999999 and verify that negative timestamps with non-zero sub-millisecond components round-trip without IllegalArgumentException, while existing aligned values remain correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.