[SUPPORT] Apply 'write.ignore.failed' when write data failed
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 112
Description
**Describe the problem you faced**
In Flink engine, if exception occurs when task writing data, it will be ignored and the exception will be reported to StreamWriteCoordinator with write event, StreamWriteCoordinator will decide whether to commit when there is write failure according to 'write.ignore.failed'.
Whether we should apply 'write.ignore.failed' ahead when write failure occurs, thus throw an exception faster.
**To Reproduce**
**Expected behavior**
1. promote 'write.ignore.failed' to hudi-client-common
2. throw exception in write handles
3. default value of 'write.ignore.failed' is false
**Environment Description**
**Additional context**
related PR:https://github.com/apache/hudi/pull/12150
**Stacktrace**
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the Flink engine write handles and StreamWriteCoordinator described in the issue, then review the related PR #12150. Verify how write failures are reported and how `write.ignore.failed` is currently read; done means the setting is available in hudi-client-common, failures are handled there, and its default is false.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100