[Bug] Comparison of timestamp types, array out of bounds
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
0.9
### Compute Engine
spark
### Minimal reproduce step
Execution failure SQL : select * from ods_mdtrt_d where updt_time >= TIMESTAMP '2025-07-10 17:00:00' limit 10;
```
Caused by: java.lang.ArrayIndexOutOfBoundsException: 49153
at org.apache.paimon.memory.MemorySegmentUtils.getLongMultiSegments(MemorySegmentUtils.java:774)
at org.apache.paimon.memory.MemorySegmentUtils.getLong(MemorySegmentUtils.java:764)
at org.apache.paimon.memory.MemorySegmentUtils.readTimestampData(MemorySegmentUtils.java:1060)
at org.apache.paimon.data.BinaryRow.getTimestamp(BinaryRow.java:323)
at org.apache.paimon.utils.InternalRowUtils.get(InternalRowUtils.java:171)
at org.apache.paimon.predicate.LeafPredicate.test(LeafPredicate.java:100)
at org.apache.paimon.predicate.And.test(And.java:55)
at org.apache.paimon.predicate.CompoundPredicate.test(CompoundPredicate.java:59)
at org.apache.paimon.operation.KeyValueFileStoreScan.filterByValueFilter(KeyValueFileStoreScan.java:197)
at org.apache.paimon.operation.KeyValueFileStoreScan.filterWholeBucketAllFiles(KeyValueFileStoreScan.java:186)
at org.apache.paimon.operation.KeyValueFileStoreScan.filterWholeBucketByStats(KeyValueFileStoreScan.java:164)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
at org.apache.paimon.operation.AbstractFileStoreScan.doPlan(AbstractFileStoreScan.java:336)
at org.apache.paimon.operation.AbstractFileStoreScan.plan(AbstractFileStoreScan.java:225)
at org.apache.paimon.table.source.snapshot.SnapshotReaderImpl.read(SnapshotReaderImpl.java:259)
at org.apache.paimon.table.source.snapshot.FullStartingScanner.scan(FullStartingScanner.java:54)
at org.apache.paimon.table.source.DataTableBatchScan.plan(DataTableBatchScan.java:76)
at org.apache.paimon.spark.PaimonBaseScan.getOriginSplits(PaimonBaseScan.scala:101)
at org.apache.paimon.spark.PaimonBaseScan.lazyInputPartitions(PaimonBaseScan.scala:110)
at org.apache.paimon.spark.PaimonBaseScan.toBatch(PaimonBaseScan.scala:121)
```
The following SQL execution was successful:
select * from ods_mdtrt_d WHERE from_unixtime(unix_timestamp(updt_time, 'yyyy-MM-dd HH:mm: ss')) >= '2025-07-10 17:00:00' limit 10;
### What doesn't meet your expectations?
-
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the Spark SQL timestamp comparison and start with MemorySegmentUtils.java, BinaryRow.java, LeafPredicate.java, and KeyValueFileStoreScan.java from the stack trace. Compare the failing direct predicate with the successful converted-timestamp query, then verify that the original query completes without an ArrayIndexOutOfBoundsException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100