apache / apache/datafusion

Expression simplifier treats arithmetic negation as bitwise NOT

Open
#24,665 1 comment 0 reactions 1 assignee Claimed by @Amogh-2404 View on GitHub
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

`Expr::Negative` represents arithmetic negation, but the expression simplifier applies bitwise-NOT identities to it. This produces wrong results for expressions combining unary minus with bitwise AND, OR, or XOR.

The same path also distributes unary minus through bitwise expressions and removes nested negation without considering the expression type.

### To Reproduce

```sql
SELECT i, (-i) & i, i | (-i), (-i) ^ i
FROM (VALUES (5), (6)) AS t(i);
```

Current result:

```text
5 0 -1 -1
6 0 -1 -1
```

Expected result:

```text
5 1 -1 -2
6 2 -2 -4
```

Generic double-negation removal can also hide overflow for checked types:

```sql
SELECT -(-arrow_cast(
CAST(-9223372036854775808 AS BIGINT),
'Timestamp(Nanosecond, None)'
));
```

The inner timestamp negation should report arithmetic overflow.

### Expected behavior

Arithmetic negation should not use bitwise-complement identities.

Double negation may be cancelled only where negation is total and involutive: signed integers with wrapping semantics, floating-point values, valid decimals, and `NULL`. Timestamp and interval negation must remain explicit because it can overflow.

### Additional context

Disabling the optimizer produces the correct bitwise results and preserves the timestamp overflow.

Signed integer scalar and array execution must use the same wrapping semantics. Pruning through signed negation must also account for the discontinuity at `MIN`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.