[Feature] Support nearest-previous partition fallback for chain table when queried partition does not exist
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Motivation
Currently, when querying a Paimon chain table table (e.g., time-series data partitioned by dt), if the queried partition does not exist, the query returns empty results. This is the default behavior.
In some scenarios , users expect the query to transparently fall back to the nearest previous existing partition rather than returning empty data.
### Solution
1.If the queried partition does not exist in the table
2.The system automatically finds the nearest previous partition (the largest partition value that is smaller than the queried one)
3.Returns data from that partition instead of empty results
## example
Table partitions: `[2024-07-01, 2024-07-02, 2024-07-05]`
snapshot paritions : `[2024-07-01]`
delta partitionss : `[2024-07-02, 2024-07-05]`
Query1 :
`SELECT * FROM table WHERE dt = '2024-07-04'`
Current behavior: returns empty
With feature enabled: returns data from `dt = '2024-07-02'` , and the value of dt column replaced with `2024-07-04`
Query2 :
`SELECT * FROM table WHERE dt = '2024-07-10'`
Current behavior: returns empty
With feature enabled: returns data from `dt = '2024-07-10' `, and the value of dt column replaced with `2024-07-10`
only work for query like : `partition column` = 'some value'
not work for query like : `partition column` <='some value' or `partition column` in ('some value')
### Anything else?
I would like to ask whether this feature is considered reasonable from the project's design perspective. If the maintainers agree that this is a valuable addition and the proposed approach is acceptable, I have already implemented the core logic locally and am ready to clean up the code and submit a PR.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue does not name implementation files, entry points, or tests. Start by locating the chain-table partition lookup and query handling for equality predicates, then verify the behavior for missing partitions, nearest-previous selection, and replacement of the returned partition value. Done means the feature's scope and design are accepted and covered by tests, including the stated exclusions for <= and IN predicates.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100