[HoodieStreamer] Register SparkWriteErrorValidator as default pre-commit validator
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
## Summary
As part of #18765, `HoodieStreamerWriteStatusValidator`'s error-check logic was migrated into explicit pre-commit orchestration steps in `StreamSync`, and a new `SparkWriteErrorValidator` was added to the validator framework as an opt-in.
The next step is to register `SparkWriteErrorValidator` as a default validator for `HoodieStreamer`, making it the canonical path.
## Before doing so, we must ensure:
1. The hot path (no-validator config) does not error out due to default registration.
2. The Step 4 inline gate in `StreamSync` can then be retired once the default is stable.
## Target
Next minor release.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the validator framework and the StreamSync orchestration changes from #18765, focusing on SparkWriteErrorValidator and the Step 4 inline gate. Verify that registering it by default does not break the no-validator hot path. Done means HoodieStreamer uses the default validator safely, with the inline gate ready to be retired once stability is confirmed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering, stream-processing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100