apache / apache/pinot

Bloom filter doesn't work on data types that have different string representations for the same value

Open
#9,899 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

When creating the bloom filter, we use the default string representation of the values to construct the bloom filter. When querying the bloom filter, we take the string from the query and use it to query the bloom filter. In order to make it work, we need to ensure the value to string conversion is consistent on the creator side and the query side.

Some data types to take extra care:
- FLOAT, DOUBLE, BIG_DECIMAL: either not support them, or handle the case of `value = 1` where the creator will put `1.0`
- BOOLEAN: bloom filter on boolean doesn't make a lot of sense, so we can drop the support for it
- TIMESTAMP: we should probably always use long value to match it, and handle `vale = '2022-12-01 01:01:01.000'` properly. Also it is not supported in `BloomFilterHandler` as of now.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.