influxdata / influxdata/influxdb

filtering differences in flux vs storage

Open
#13,880 8 comments 0 reactions 1 assignee View on GitHub

@aanthony1243 is already working on this.

Since May 10, 2019.

breaking change difficulty/medium team/query
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

Currently, there are some mismatches in how the flux filter function filters rows/tables and how the storage level filters them. There are currently two issues where this can create confusion for the user.

  1. When storage finds that a filter results in an empty table, nothing is emitted. In Flux, an empty table will be emitted: https://github.com/influxdata/flux/issues/45
  2. Storage interprets r.TAG == "" as meaning either that the tag is not present on the table, or else the tag is present but set to the empty string. Similarly, r.TAG != "" effectively filters tables that are missing that tag. In Flux, there is a distinction between a column that has an empty string for a value and a column that does not exist on a particular table. This has manifested here: https://github.com/influxdata/flux/issues/1206. In particular, the user will observe different behavior based on whether the planner is able to match a rule and push the filter down to storage, or it is unable and does filtering on the query engine side.

This issue is here to generate some discussion on these two points in order to work out a consistent solution that works well for both flux --- which is meant to be source-agnostic and influxdb --- which has a specific data model interpretation.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.