apache / apache/datafusion

Potential performance regression with comparisions to scalar values

Open
#14,291 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

There is concern changes in this PR will cause regression (as it may convert some numbers to Decimal128 rather than more efficient Integer)
- https://github.com/apache/datafusion/pull/14223

However, it is not clear (to me at least) exactly the implications of the change

@berkaysynnada has found a potential performance regression, as described here: https://github.com/apache/datafusion/pull/14223#issuecomment-2612423408

cc @nuno-faria @jonahgao and @berkaysynnada and @ozankabak

### To Reproduce

_No response_

### Expected behavior

I want to make sure that this issue is resolved before we release datafusion 45.0.0, so filing this ticket so we don't forget

Examples of ways to resolve the ticket:
- Decide the change was fine
- Revert the changes
- Others

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reading PR #14223 and the linked issue comment describing the potential regression. Determine the performance implications of converting scalar values to Decimal128 rather than Integer, then establish whether the change should remain, be reverted, or otherwise resolved before the DataFusion 45.0.0 release.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.