apache / apache/datafusion

Extend `datafusion-spark` `sequence` function

Open
#22,008 1 comment 0 reactions 1 assignee Claimed by @erenavsarogullari View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

This is follow-up issue for datafusion-spark sequence function: https://github.com/apache/datafusion/pull/20955#discussion_r3178918596

Following cases also need to be supported:
Currently, following existing test cases having Interval (step) definition for both TIMESTAMP and DATE work as expected:
https://github.com/apache/datafusion/pull/20955/changes#diff-f38c343ca9807a82465bea986e7bd1417947b26407dbd82e91ed328b6c15500dR134

However, the problem occurs when Interval (step) definition is not set as follows:
**Spark:**
```
spark.sql("SELECT sequence(TIMESTAMP'2023-01-03T00:00:00', TIMESTAMP'2023-01-01T00:00:00');")
[ArraySeq(2023-01-03 00:00:00.0, 2023-01-02 00:00:00.0, 2023-01-01 00:00:00.0)]

spark.sql("SELECT sequence(DATE'2018-01-03', DATE'2018-01-01');")
[ArraySeq(2018-01-03, 2018-01-02, 2018-01-01)]
```

**DataFusion:**
```
SELECT sequence(DATE '2018-01-04', DATE '2018-01-01');
actual: []
expected: [2018-01-04, 2018-01-03, 2018-01-02, 2018-01-01]

SELECT sequence(TIMESTAMP '2023-01-03T00:00:00', TIMESTAMP '2023-01-01T00:00:00');
actual: []
expected: [2023-01-03T00:00:00, 2023-01-02T00:00:00, 2023-01-01T00:00:00]
```

### Describe the solution you'd like

Has been explained at the first section.

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.