[Feature] pypaimon supports overwrite writes.
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Motivation
In offline model inference and table writing scenarios, it is common to conduct multiple offline evaluation experiments. It is necessary to support overwrite writes so that the results from the last run are retained. Currently, this is not supported in the code. For example:
```python
def overwrite(self, partition, commit_messages: List[CommitMessage], commit_identifier: int):
"""Commit the given commit messages in overwrite mode."""
raise RuntimeError("overwrite unsupported yet")
```
### Solution
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating pypaimon's overwrite method shown in the issue and tracing the table-writing and commit paths around it. Determine how overwrite should retain results from the latest evaluation run, then identify the relevant tests or add coverage for the overwrite behavior; done means overwrite writes no longer raise the unsupported error and the expected previous results are replaced.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, python
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100