NVIDIA / NVIDIA/cudf

[BUG/pandas-compat]: Handling of type promotion and division/mod by zero for boolean columns

Open
#12,162 0 comments 0 reactions 0 assignees View on GitHub
0 - Backlog bug Python
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

After #12074, most type promotions between columns of mixed types (and non-mixed types) match pandas. The exception is columns with boolean dtypes.

Pandas have taken the decision to disallow division and exponentiation on boolean types when both operands are booleans (https://github.com/pandas-dev/pandas/blob/d13c9e034ce8a1d738766c4b1cec80c76f5523be/pandas/core/ops/array_ops.py#L503).

Aside: I kind of disagree with this since this is all perfectly well defined (excepting the usual caveat of division by zero).

When only one of the operands is `bool`, the status quo depends on the dtype of the other operand:

## Pandas behaviour:

For `a % b`, with `a == 1`, `b == 0` for various dtypes

| dtype-a \ dtype-b | bool | int | float |
|----------|------|------|-|
| bool | int8(0) (or ZeroDivisionError[^1]) | float64(NaN) | float64(NaN) |
| int | int64(0) (or ZeroDivisionError[^1]) | float64(NaN) | float64(NaN) |
| float | float64(NaN) (or ZeroDivisionError[^1])| float64(NaN) | float64(NaN) |

For `a / b` (or `a // b`) with `a == 1`, `b = 0`

| dtype-a \ dtype-b | bool | int | float |
|----------|------|------|-|
| bool | NotImplemented (or ZeroDivisionError[^1]) | float64(inf) | float64(inf) |
| int | float64(inf)(or ZeroDivisionError[^1]) | float64(inf) | float64(inf) |
| float | float64(inf) (or ZeroDivisionError[^1])| float64(inf) | float64(inf) |

For `a % b`, with `a == 0`, `b == 0` for various dtypes

| dtype-a \ dtype-b | bool | int | float |
|----------|------|------|-|
| bool | int8(0)(or ZeroDivisionError[^1]) | float64(NaN) | float64(NaN) |
| int | int64(0) (or ZeroDivisionError[^1])| float64(NaN) | float64(NaN) |
| float | float64(NaN) (or ZeroDivisionError[^1])| float64(NaN) | float64(NaN) |

For `a / b` (or `a // b`) with `a == 0`, `b = 0`

| dtype-a \ dtype-b | bool | int | float |
|----------|------|------|-|
| bool | NotImplemented (or ZeroDivisionError[^1]) | float64(NaN) | float64(NaN) |
| int | float64(NaN) (or ZeroDivisionError[^1]) | float64(NaN) | float64(NaN) |
| float | float64(NaN) (or ZeroDivisionError[^1]) | float64(NaN) | float64(NaN) |

[^1]: If the operands are different lengths, we get a ZeroDivisionError (see https://github.com/pandas-dev/pandas/issues/49699)

## cuDF behaviour:

For `a % b`, with `a == 1`, `b == 0` for various dtypes

| dtype-a \ dtype-b | bool | int | float |
|----------|------|------|-|
| bool | bool(0) | float64(NaN) | float64(NaN) |
| int | int64(2**32 - 1) | float64(NaN) | float64(NaN) |
| float | float64(NaN) | float64(NaN) | float64(NaN) |

For `a // b` with `a == 1`, `b = 0`

| dtype-a \ dtype-b | bool | int | float |
|----------|------|------|-|
| bool | bool(1) | float64(inf) | float64(inf) |
| int | int64(2**32 - 1) | float64(inf) | float64(inf) |
| float | float64(inf) | float64(inf) | float64(inf) |

For `a / b` with `a == 1`, `b = 0`

| dtype-a \ dtype-b | bool | int | float |
|----------|------|------|-|
| bool | float32(inf) | float64(inf) | float64(inf) |
| int | float64(inf) | float64(inf) | float64(inf) |
| float | float64(inf) | float64(inf) | float64(inf) |

For `a % b`, with `a == 0`, `b == 0` for various dtypes

| dtype-a \ dtype-b | bool | int | float |
|----------|------|------|-|
| bool | bool(0) | float64(NaN) | float64(NaN) |
| int | int64(2**32 - 1) | float64(NaN) | float64(NaN) |
| float | float64(NaN) | float64(NaN) | float64(NaN) |

For `a // b` with `a == 0`, `b = 0`

| dtype-a \ dtype-b | bool | int | float |
|----------|------|------|-|
| bool | bool(False) | float64(NaN) | float64(NaN) |
| int | int64(2**32 - 1) | float64(NaN) | float64(NaN) |
| float | float64(NaN) | float64(NaN) | float64(NaN) |

For `a / b` with `a == 0`, `b = 0`

| dtype-a \ dtype-b | bool | int | float |
|----------|------|------|-|
| bool | float32(NaN) | float64(NaN) | float64(NaN) |
| int | float64(NaN) | float64(NaN) | float64(NaN) |
| float | float64(NaN) | float64(NaN) | float64(NaN) |

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.