[C++] case when expression divide overflow.
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the bug, including details regarding any error messages, version, and platform.
when we execute casewhen expression, such as:
CASE WHEN i > 0 THEN j / i ELSE j END
If column i has values( 0 ), we will get division overflow problem, j / 0 -> overflow。
Why does this problem occur?
I read the source code and found that the masks for these conditions are calculated separately, and the previous mask is not used to participate in subsequent operations. This will cause the subsequent division to overflow.
Currently, we have two solutions.
1. Regardless of performance
Using the previous mask to participate in subsequent operations will cause performance degradation in all cases.
```cpp
for (size_t i = 0; i < arguments.size(); ++i) {
// get SelectorVector from arguments before computed;
std::shared_ptr filtered_mask =
GetExpressionMask(call->function_name, arguments, i);
if (filtered_mask != nullptr) {
auto values = input.values;
// construct filtered batch
for (auto& value : values) {
if (value.is_array()) {
std::vector invert_args;
invert_args.push_back(filtered_mask);
ARROW_ASSIGN_OR_RAISE(auto null_mask,
CallFunction("invert", invert_args, exec_context));
ARROW_ASSIGN_OR_RAISE(
value, ReplaceWithMask(value, null_mask, MakeNullScalar(value.type())));
}
}
ARROW_ASSIGN_OR_RAISE(auto new_batch, ExecBatch::Make(std::move(values)));
// execute and pass child expr
ARROW_ASSIGN_OR_RAISE(
arguments[i],
ExecuteScalarExpression(call->arguments[i], new_batch, exec_context));
} else {
ARROW_ASSIGN_OR_RAISE(
arguments[i], ExecuteScalarExpression(call->arguments[i], input, exec_context));
}
ARROW_ASSIGN_OR_RAISE(
arguments[i], ExecuteScalarExpression(call->arguments[i], input, exec_context));
}
```
2. Considering performance, only division expressions are treated specially.
We currently apply this method to the application layer, which constructs a complex expression to avoid division overflow problems.
case when j > 0 then i / j else i end
->
case when j > 0 then i / ((case when j > 0 then j else null end)) else i end
**Based on the above background, does the community have any good ways to fix this problem?**
### Component(s)
C++
Contributor guide
Assessment
This issue has not been assessed yet.