opensearch-project / opensearch-project/sql
[FEATURE] Support "consecutive=true" argument in `dedup` command with Calcite
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 176
- Forks
- 229
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 43
Description
Is your feature request related to a problem?
Currently, we implements dedup by row number window function. The argument "consecutive=true" required the input data forced with certain ordering.
For example, only following ppl have business meaning.
source=t | sort timestamp | dedup 2 col1 consecutive=true
or
source=t | dedup 2 col1 consecutive=true sort by timestamp
query without sorting, such as
source=t | dedup 2 col1 consecutive=true
should throw exception "dedupe data with consecutive has no ordering"
What solution would you like?
A clear and concise description of what you want to happen.
What alternatives have you considered?
A clear and concise description of any alternative solutions or features you've considered.
Do you have any additional context?
Add any other context or screenshots about the feature request here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the Calcite-backed planning path for the dedup command and reviewing how its row-number window and input ordering are handled. Find or add relevant dedup and ordering tests; done means consecutive=true works with an explicit ordering and an unordered query raises the specified exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100