Test coverage for table creation
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
[follow up of
https://github.com/apache/hudi/pull/12615|https://github.com/apache/hudi/pull/12615]
goal for this patch (open to splitting it up into multiple):
* for new table creation, we should default to table version 8.
* for new table creation, if user overrides "hoodie.write.table.version", we will honor the overridden value.
We need to validate and may be fix all writers (spark-ds, spark-sql, deltastreamer and spark structured streaming ingestion)
|Deltastreamer :
[hudi/hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/StreamSync.java|https://github.com/apache/hudi/blob/ef5b9ca3777775fae29c04c06cbb743183e9eef3/hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/StreamSync.java#L421]
Line 421 in [ef5b9ca|https://github.com/apache/hudi/commit/ef5b9ca3777775fae29c04c06cbb743183e9eef3]
||HoodieTableMetaClient initializeEmptyTable(HoodieTableMetaClient.TableBuilder tableBuilder, String partitionColumns,|
[hudi/hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java|https://github.com/apache/hudi/blob/ef5b9ca3777775fae29c04c06cbb743183e9eef3/hudi-utilities/src/test/java/org/apache/hudi/utilities/deltastreamer/TestHoodieDeltaStreamer.java#L428]
Line 428 in [ef5b9ca|https://github.com/apache/hudi/commit/ef5b9ca3777775fae29c04c06cbb743183e9eef3]
||public void testTableCreation() throws Exception {|
spark-sql and spark-ds: guess this patch already have tests.
spark-streaming:
[hudi/hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestStructuredStreaming.scala|https://github.com/apache/hudi/blob/ef5b9ca3777775fae29c04c06cbb743183e9eef3/hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/hudi/functional/TestStructuredStreaming.scala#L208]
Line 208 in [ef5b9ca|https://github.com/apache/hudi/commit/ef5b9ca3777775fae29c04c06cbb743183e9eef3]
||def testStructuredStreaming(tableType: HoodieTableType): Unit = {|
|
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-8914
- Type: Test
- Fix version(s):
- 1.1.0
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.