apache / apache/datafusion

date_part returning wrong results due to overflows

Open Beginner friendly
#14,738 7 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

When playing with the date_part function, I see that there's ways of triggering int32 multiplication overflows that either panic on a debug build, or return the wrong number at runtime.

### To Reproduce

Executing the following statement shows the behavior.

```sql
SELECT date_part('microsecond', timestamp '1970-01-01T00:40:00' - timestamp '1970-01-01T00:00:00')
```

[DataFusion fiddle link](https://datafusion-fiddle.vercel.app?q=eyJkZGwiOiJcbiIsInNlbGVjdCI6IlNFTEVDVCBkYXRlX3BhcnQoJ21pY3Jvc2Vjb25kJywgdGltZXN0YW1wICcxOTcwLTAxLTAxVDAwOjQwOjAwJyAtIHRpbWVzdGFtcCAnMTk3MC0wMS0wMVQwMDowMDowMCcpXG4ifQ==) <- returns a wrong random number

[Postgres fiddle link](https://www.db-fiddle.com/f/4jyoMCicNSZpjMt4jFYoz5/16149) <- returns 0

### Expected behavior

The date_part function should behave the same as Postgres

### Additional context

Not 100% sure, but I would say that the changes introduced in https://github.com/apache/datafusion/pull/13466 look suspicious. There, the inner calculations are using int32 types, which are easy to overflow. Special mention to this line of code:
https://github.com/gabotechs/datafusion/blob/763bd681f09d58ce285ab3a677b81291c41adfce/datafusion/functions/src/datetime/date_part.rs#L290-L290

Contributor guide

Open the contributing guide

Research direction

Start by running the supplied date_part SQL query and inspect datafusion/functions/src/datetime/date_part.rs, especially the calculation around line 290. Trace the integer types involved and add regression coverage for the reproducer; done means the query returns the same result as PostgreSQL without overflow or panic.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.