apache / apache/druid

Bound filter sorting direction

Open
#9,382 0 comments 6 reactions 0 assignees View on GitHub
Design Review Proposal
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Motivation

Changing the direction of sorting (ascending/descending) using the [bound filter](https://druid.apache.org/docs/latest/querying/filters.html#bound-filter) seems to be a natural consequence of the sorting functionality, yet it is not possible to change it.

### Proposed changes

The most obvious change in the bound filter API would be the addition of a parameter. I can see two ways of implementing that:

- `orderingDirection` (string), which would take one of two values: `'ascending'` and `'descending'` (`'ascending'` as default)
- `ascendingOrder` (boolean), which would imply the reversed ordering (`true` as default)

Obviously, documentation update is needed as well.

### Rationale

The advantage of the solution described above is it's simplicity in the API and easy maintenance in the future. Yet, after vague analysis of the code, I think that it would require major changes in the code and would not be a simple functionality addition, but I might see it wrong (please, correct me if it is like so).

Each comparator in [StringComparators has it's own cache ID](https://github.com/apache/druid/blob/master/processing/src/main/java/org/apache/druid/query/ordering/StringComparators.java#L46). If I understand it correctly, to handle reversed comparators correctly, the IDs for them would have to be added, so a way of distinguishing them is required. The addition of new comparators could be a simple and non-invasive change - all existing ordering options would remain unchanged and each of them would have a reversed one. Simple "reverse" flag as described above would probably be harder to implement in regard of the cache.

In the API this would require expanding the `ordering` options with: `lexicographic_reversed`, `alphanumeric_reversed`, `numeric_reversed`, `strlen_reversed`, `version_reversed`.

### Operational impact

- Is anything going to be deprecated or removed by this change? How will we phase out old behavior?
- The default behavior for both lack of the "reverse" flag and already existing values would remain the same.
- Is there a migration path that cluster operators need to be aware of?
- No migration is needed.
- Will there be any effect on the ability to do a rolling upgrade, or to do a rolling _downgrade_ if an operator wants to switch back to a previous version?
- No.

Contributor guide

Open the contributing guide

Research direction

Start with the bound-filter API and documentation, then inspect processing/src/main/java/org/apache/druid/query/ordering/StringComparators.java, especially its comparator cache IDs. Resolve whether reversed comparator options or a direction parameter fits the API and cache design. Done means bound filters support descending ordering while existing defaults remain unchanged and the documentation describes the new options.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
database
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.