apache / apache/beam

Beam SQL operator side condition validation is sensitive to optimization

Open
#19,028 0 comments 0 reactions 0 assignees View on GitHub
bug dsl P3 sql
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Beam SQL converts Calcite's operators to Beam SQL operators post-optimization, when the final rels are being converted to a pipeline. This means that the optimization process can obscure the ability to validate things easily.

Specifically, for a `CEIL(date TO DAY)` we only support rounding to month or year (because of the underlying Calcite implementation. We want to be able to get the `DAY` and validate it before running. Today this is not validated properly because it is inconvenient.

Before optimization, we can check that the second operand is a symbol and see what it is. After optimization, the conversion to Calc hides it behind a LocalRef. We could look up the LocalRef but it would be cleaner to avoid evaluation-like logic during type checking/validation phase.

Imported from Jira [BEAM-4621](https://issues.apache.org/jira/browse/BEAM-4621). Original Jira may contain additional context.
Reported by: kenn.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.