apache / apache/arrow

[C++] case when expression divide overflow.

Open
#37,765 2 comments 0 reactions 0 assignees View on GitHub
Component: C++ Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

when we execute casewhen expression, such as:

CASE WHEN i > 0 THEN j / i ELSE j END

If column i has values( 0 ), we will get division overflow problem, j / 0 -> overflow。

Why does this problem occur?

I read the source code and found that the masks for these conditions are calculated separately, and the previous mask is not used to participate in subsequent operations. This will cause the subsequent division to overflow.

Currently, we have two solutions.

1. Regardless of performance

Using the previous mask to participate in subsequent operations will cause performance degradation in all cases.

```cpp
for (size_t i = 0; i < arguments.size(); ++i) {
// get SelectorVector from arguments before computed;
std::shared_ptr filtered_mask =
GetExpressionMask(call->function_name, arguments, i);
if (filtered_mask != nullptr) {
auto values = input.values;
// construct filtered batch
for (auto& value : values) {
if (value.is_array()) {
std::vector invert_args;
invert_args.push_back(filtered_mask);
ARROW_ASSIGN_OR_RAISE(auto null_mask,
CallFunction("invert", invert_args, exec_context));
ARROW_ASSIGN_OR_RAISE(
value, ReplaceWithMask(value, null_mask, MakeNullScalar(value.type())));
}
}
ARROW_ASSIGN_OR_RAISE(auto new_batch, ExecBatch::Make(std::move(values)));
// execute and pass child expr
ARROW_ASSIGN_OR_RAISE(
arguments[i],
ExecuteScalarExpression(call->arguments[i], new_batch, exec_context));
} else {
ARROW_ASSIGN_OR_RAISE(
arguments[i], ExecuteScalarExpression(call->arguments[i], input, exec_context));
}
ARROW_ASSIGN_OR_RAISE(
arguments[i], ExecuteScalarExpression(call->arguments[i], input, exec_context));
}
```

2. Considering performance, only division expressions are treated specially.

We currently apply this method to the application layer, which constructs a complex expression to avoid division overflow problems.

case when j > 0 then i / j else i end
->
case when j > 0 then i / ((case when j > 0 then j else null end)) else i end

**Based on the above background, does the community have any good ways to fix this problem?**

### Component(s)

C++

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.