Apply 'write.ignore.failed' when write data failed
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-8400
- Type: Improvement
- Fix version(s):
- 1.2.0
---
## Comments
30/Oct/24 06:42;fhan688;{quote}We should clarify these items:
# should we promote the {{write.ignore.failed}} option to a common write config for each engine? Previously each eagine has it's own options and behavior.
# should we throw the exception in write handles or in the driver(after the write status are collected);
# should this option by default false or true?{quote}
# I agree. {{write.ignore.failed}} is a config in FlinkOptions and we promote it to HoodieWriteConfig in hudi-client-common module and named 'hoodie.write.ignore.failed' in this PR.
# I think fast fail is a better choice. In heavy PRD traffic job, several minutes late means huge amounts of records needs to be dealt with when restore.
# false, consider data quality.;;;
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.