[SUPPORT] What Class Name to use for hoodie.errortable.write.class
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
I'm trying out Hudi error tables, but I'm having trouble finding the documentation for the hoodie.errortable.write.class value. Could you please assist me?
# sample config
```
hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.SimpleKeyGenerator
hoodie.datasource.write.recordkey.field=invoiceid
hoodie.datasource.write.partitionpath.field=destinationstate
hoodie.streamer.source.dfs.root=file:///Users/soumilshah/IdeaProjects/SparkProject/apache-hudi-delta-streamer-labs/E1/sampledata/
hoodie.datasource.write.precombine.field=replicadmstimestamp
hoodie.streamer.transformer.sql=SELECT * FROM a where sas
hoodie.errortable.base.path=file:///Users/soumilshah/IdeaProjects/SparkProject/apache-hudi-delta-streamer-labs/E1/error/
hoodie.errortable.target.table.name=error_invoice
hoodie.errortable.enable=true
hoodie.errortable.write.class=
```
# Job
```
spark-submit \
--class org.apache.hudi.utilities.streamer.HoodieStreamer \
--packages org.apache.hudi:hudi-spark3.4-bundle_2.12:0.14.0 \
--properties-file spark-config.properties \
--master 'local[*]' \
--executor-memory 1g \
/Users/soumilshah/IdeaProjects/SparkProject/apache-hudi-delta-streamer-labs/E1/jar/hudi-utilities-slim-bundle_2.12-0.14.0.jar \
--table-type COPY_ON_WRITE \
--op UPSERT \
--transformer-class org.apache.hudi.utilities.transform.SqlQueryBasedTransformer \
--source-ordering-field replicadmstimestamp \
--source-class org.apache.hudi.utilities.sources.ParquetDFSSource \
--target-base-path file:///Users/soumilshah/IdeaProjects/SparkProject/apache-hudi-delta-streamer-labs/E1/silver/ \
--target-table invoice \
--props hudi_tbl.props
```
I want to purposely fail the job and I want to see error tables being created
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the `hoodie.errortable.write.class` configuration in the sample properties and the `HoodieStreamer` entry point shown in the job command. Determine which class name is supported and document how intentional job failures produce error tables; the issue does not identify a documentation file or test to update.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100