[Feature] Rewrite_file_index action job add partitions option
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Motivation
The `rewrite_file_index` Procedure supports specifying partitions, but in internal production practice, the Flink Procedure syntax cannot be used. Only the `rewrite_file_index` Action job can be used, however, it was found that `rewrite_file_index` Action job does not support partition specification, so this capability will be aligned with the Procedure.
### Solution
Add partition parameters
### Anything else?
It was found that although the `RewriteFileIndexActionITCase` unit test ran successfully, it did not achieve the desired testing effect; specifically, no test data or index files were generated. The main reason is that the job for constructing test data and the `rewrite_file_index` job are not executed synchronously.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the rewrite_file_index Action job and its RewriteFileIndexActionITCase test, then compare the Action job's arguments with the rewrite_file_index Procedure's partition support. Verify that test data and index files are created before the rewrite job runs; done means partition parameters work and the integration test exercises the generated index files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100