apache / apache/gluten

[VL] PartialProject export Timestamp overflow

Open
#11,985 0 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

```bash
Caused by: org.apache.gluten.exception.GlutenException: Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: Operator::getOutput failed for [operator: ValueStream, plan node ID: 0]: Error during calling Java code from native code: org.apache.gluten.exception.GlutenException: Exception: VeloxUserError
Error Source: USER
Error Code: INVALID_ARGUMENT
Reason: Could not convert Timestamp(140514369963968, 140514371047552) to microseconds, Exception: VeloxUserError
Error Source: USER
Error Code: ARITHMETIC_ERROR
Reason: integer overflow: 140514369963968 * 1000000
Retriable: False
Function: checkedMultiply
File: /home/jenkins/agent/workspace/di-spark/Gluten/CI_spark_gluten/ep/build-velox/build/velox_ep/./velox/common/base/CheckedArithmetic.h
Line: 51
Stack trace:
# 0 _ZN8facebook5velox7process10StackTraceC1Ei
# 1 _ZN8facebook5velox14VeloxExceptionC1EPKcmS3_St17basic_string_viewIcSt11char_traitsIcEES7_S7_S7_bNS1_4TypeES7_
# 2 _ZN8facebook5velox6detail14veloxCheckFailINS0_14VeloxUserErrorERKSsEEvRKNS1_18VeloxCheckFailArgsET0_
# 3 _ZNK8facebook5velox9Timestamp8toMicrosEv
# 4 _ZN8facebook5velox12_GLOBAL__N_125gatherFromTimestampBufferERKNS0_10BaseVectorERKNS1_9SelectionE13TimestampUnitRNS0_6BufferE
# 5 _ZN8facebook5velox12_GLOBAL__N_112exportValuesERKNS0_10BaseVectorERKNS1_9SelectionERK12ArrowOptionsR10ArrowArrayPNS0_6memory10MemoryPoolERNS1_24VeloxToArrowBridgeHolderE.constprop.0
# 6 _ZN8facebook5velox12_GLOBAL__N_117exportToArrowImplERKNS0_10BaseVectorERKNS1_9SelectionERK12ArrowOptionsR10ArrowArrayPNS0_6memory10MemoryPoolE
# 7 _ZN8facebook5velox12_GLOBAL__N_117exportToArrowImplERKNS0_10BaseVectorERKNS1_9SelectionERK12ArrowOptionsR10ArrowArrayPNS0_6memory10MemoryPoolE
# 8 _ZN8facebook5velox12_GLOBAL__N_117exportToArrowImplERKNS0_10BaseVectorERKNS1_9SelectionERK12ArrowOptionsR10ArrowArrayPNS0_6memory10MemoryPoolE
# 9 _ZN8facebook5velox13exportToArrowERKSt10shared_ptrINS0_10BaseVectorEER10ArrowArrayPNS0_6memory10MemoryPoolERK12ArrowOptions
# 10 _ZN6gluten18VeloxColumnarBatch16exportArrowArrayEv
# 11 Java_org_apache_gluten_columnarbatch_ColumnarBatchJniWrapper_exportToArrow
# 12 0x00007fccfd0186e7

Retriable: False
Function: toMicros
File: /home/jenkins/agent/workspace/di-spark/Gluten/CI_spark_gluten/ep/build-velox/build/velox_ep/./velox/type/Timestamp.h
Line: 195
```

### Gluten version

main branch

### Spark version

None

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

```bash

```

Contributor guide

Open the contributing guide

Research direction

Start with Velox's Timestamp::toMicros in velox/type/Timestamp.h and the timestamp export path identified in the stack trace, including gatherFromTimestampBuffer and exportToArrow. Reproduce the PartialProject export with the timestamp value that triggers the overflow, then verify the export handles the case without the reported arithmetic error.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, scala
Domain
backend, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.