[VL] PartialProject export Timestamp overflow
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Backend
VL (Velox)
### Bug description
```bash
Caused by: org.apache.gluten.exception.GlutenException: Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: Operator::getOutput failed for [operator: ValueStream, plan node ID: 0]: Error during calling Java code from native code: org.apache.gluten.exception.GlutenException: Exception: VeloxUserError
Error Source: USER
Error Code: INVALID_ARGUMENT
Reason: Could not convert Timestamp(140514369963968, 140514371047552) to microseconds, Exception: VeloxUserError
Error Source: USER
Error Code: ARITHMETIC_ERROR
Reason: integer overflow: 140514369963968 * 1000000
Retriable: False
Function: checkedMultiply
File: /home/jenkins/agent/workspace/di-spark/Gluten/CI_spark_gluten/ep/build-velox/build/velox_ep/./velox/common/base/CheckedArithmetic.h
Line: 51
Stack trace:
# 0 _ZN8facebook5velox7process10StackTraceC1Ei
# 1 _ZN8facebook5velox14VeloxExceptionC1EPKcmS3_St17basic_string_viewIcSt11char_traitsIcEES7_S7_S7_bNS1_4TypeES7_
# 2 _ZN8facebook5velox6detail14veloxCheckFailINS0_14VeloxUserErrorERKSsEEvRKNS1_18VeloxCheckFailArgsET0_
# 3 _ZNK8facebook5velox9Timestamp8toMicrosEv
# 4 _ZN8facebook5velox12_GLOBAL__N_125gatherFromTimestampBufferERKNS0_10BaseVectorERKNS1_9SelectionE13TimestampUnitRNS0_6BufferE
# 5 _ZN8facebook5velox12_GLOBAL__N_112exportValuesERKNS0_10BaseVectorERKNS1_9SelectionERK12ArrowOptionsR10ArrowArrayPNS0_6memory10MemoryPoolERNS1_24VeloxToArrowBridgeHolderE.constprop.0
# 6 _ZN8facebook5velox12_GLOBAL__N_117exportToArrowImplERKNS0_10BaseVectorERKNS1_9SelectionERK12ArrowOptionsR10ArrowArrayPNS0_6memory10MemoryPoolE
# 7 _ZN8facebook5velox12_GLOBAL__N_117exportToArrowImplERKNS0_10BaseVectorERKNS1_9SelectionERK12ArrowOptionsR10ArrowArrayPNS0_6memory10MemoryPoolE
# 8 _ZN8facebook5velox12_GLOBAL__N_117exportToArrowImplERKNS0_10BaseVectorERKNS1_9SelectionERK12ArrowOptionsR10ArrowArrayPNS0_6memory10MemoryPoolE
# 9 _ZN8facebook5velox13exportToArrowERKSt10shared_ptrINS0_10BaseVectorEER10ArrowArrayPNS0_6memory10MemoryPoolERK12ArrowOptions
# 10 _ZN6gluten18VeloxColumnarBatch16exportArrowArrayEv
# 11 Java_org_apache_gluten_columnarbatch_ColumnarBatchJniWrapper_exportToArrow
# 12 0x00007fccfd0186e7
Retriable: False
Function: toMicros
File: /home/jenkins/agent/workspace/di-spark/Gluten/CI_spark_gluten/ep/build-velox/build/velox_ep/./velox/type/Timestamp.h
Line: 195
```
### Gluten version
main branch
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
Contributor guide
Research direction
Start with Velox's Timestamp::toMicros in velox/type/Timestamp.h and the timestamp export path identified in the stack trace, including gatherFromTimestampBuffer and exportToArrow. Reproduce the PartialProject export with the timestamp value that triggers the overflow, then verify the export handles the case without the reported arithmetic error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, scala
- Domain
- backend, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100