Support rough set filter for Decimal columns
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1k
- Forks
- 423
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 24
Description
Background
TiFlash rough set filter can use storage-level min/max metadata to skip packs before reading full column data. Decimal columns are common for monetary and metric fields, but decimal predicates are currently not handled end-to-end by the rough set filter path, so queries with selective Decimal filters cannot benefit from pack pruning.
For example:
SELECT *
FROM t
WHERE decimal_col >= 100.00 AND decimal_col < 200.00;
Without Decimal rough set support, TiFlash may need to scan more packs even when the Decimal predicate is highly selective.
Proposal
Support Decimal columns in the rough set min/max filter path.
Main scope:
- Build and persist min/max index metadata for Decimal columns.
- Read Decimal min/max metadata from DMFile and expose it to rough set checks.
- Allow Decimal column predicates to be parsed into rough set operators only when the non-null literal is encoded as
tipb::ExprType::MysqlDecimal. - Keep mixed Decimal/non-Decimal predicates conservative: parser should mark them as unsupported when statically detectable, and runtime
CanNotCompare -> Someshould remain as a safety net. - Use scale-aware Decimal comparison for Decimal32/Decimal64/Decimal128/Decimal256, including different precision/scale combinations between the predicate constant and the column/index value.
- Support comparison predicates, null-safe equality,
IN, and nullable Decimal columns where applicable.
Acceptance Criteria
- Decimal predicates can create effective rough set operators and prune packs with Decimal min/max indexes.
- Mixed Decimal/non-Decimal predicates do not create misleading rough set operators and remain conservative.
- Unit tests cover parser behavior, min/max rough check behavior, nullable Decimal behavior, and DMFile write/read of Decimal min/max indexes.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the rough set parser and checks, then trace DMFile min/max metadata write and read paths for Decimal columns. Add coverage for parser behavior, scale-aware comparisons, nullable values, mixed-type predicates, and Decimal min/max persistence. Done means selective Decimal predicates prune packs while unsupported mixed predicates remain conservative.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100