influxdata / influxdata/influxdb
filtering differences in flux vs storage
Open
@aanthony1243 is already working on this.
Since May 10, 2019.
breaking change
difficulty/medium
team/query
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
Currently, there are some mismatches in how the flux filter function filters rows/tables and how the storage level filters them. There are currently two issues where this can create confusion for the user.
- When storage finds that a filter results in an empty table, nothing is emitted. In Flux, an empty table will be emitted: https://github.com/influxdata/flux/issues/45
- Storage interprets
r.TAG == ""as meaning either that the tag is not present on the table, or else the tag is present but set to the empty string. Similarly,r.TAG != ""effectively filters tables that are missing that tag. In Flux, there is a distinction between a column that has an empty string for a value and a column that does not exist on a particular table. This has manifested here: https://github.com/influxdata/flux/issues/1206. In particular, the user will observe different behavior based on whether the planner is able to match a rule and push the filter down to storage, or it is unable and does filtering on the query engine side.
This issue is here to generate some discussion on these two points in order to work out a consistent solution that works well for both flux --- which is meant to be source-agnostic and influxdb --- which has a specific data model interpretation.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.