Stop handling more data if task is aborted & clean partial files if possible in task side
Open
engine:spark
from-jira
priority:high
status:pr-available
type:improvement
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Spark would set interrupt status in TaskContext if the task is aborted, HUDI needs to respect that to stop immediately. Also, we can clean partial files at task side to ensure these files won't be left.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-6989
- Type: Improvement
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.