[BUG] hive_style_partitioning + slash.separated.date.partitioning accepted outside SQL DDL despite the config documenting them as exclusive
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
## Bug Description
**What happened:**
The documentation of `hoodie.datasource.write.slash.separated.date.partitioning` states the
constraint outright:
> "Flag to indicate whether to use slash separated date partitioning. If set to true, date
> partition values in yyyy-MM-dd format will be transformed to yyyy/MM/dd directory structure.
> By default false. **Cannot be used together with hive-style partitioning.**"
Only SQL DDL enforces it. `HoodieCatalogTable#extraTableConfig` rejects the combination by
inspecting `sqlOptions`, but `HoodieWriterUtils`, `HoodieSparkSqlWriter` and `BaseKeyGenerator`
have no equivalent check, so `df.write.format("hudi")` and HoodieStreamer accept
`hive_style_partitioning=true` together with `slash.separated.date.partitioning=true`.
For a single partition field the two write paths then disagree, and both are live (Avro versus
Spark record type, plus the row writer):
| path | partition directory |
|---|---|
| Avro (`KeyGenUtils#getPartitionPath`) | `date_col=2026/01/05` |
| Row / InternalRow (`PartitionPathFormatterBase`) | `date_col=2026-01-05` |
The Avro layout is also not readable back:
`HoodieSparkUtils#doParsePartitionColumnValues` takes the `date_col=` prefix branch and never
restores `/` to `-`.
**What you expected:**
A documented-invalid combination should be rejected by every writer, not only by SQL DDL.
**Steps to reproduce:**
1. `df.write.format("hudi")` with both `hoodie.datasource.write.hive_style_partitioning=true` and
`hoodie.datasource.write.slash.separated.date.partitioning=true`, one date partition column.
2. The write is accepted; compare the directory produced by the Avro path against the row-writer
one, and read the Avro-written table back.
**Suggested fix:**
Add the mutual-exclusion check to `HoodieWriterUtils.validateTableConfig` so it applies to every
write path, matching what the config documentation already promises.
Raised during review of #19648.
## Environment
**Hudi version:** master (1.3.0-SNAPSHOT)
**Query engine:** Spark (`df.write` / HoodieStreamer, i.e. not SQL DDL)
**Relevant configs:** `hoodie.datasource.write.hive_style_partitioning=true` +
`hoodie.datasource.write.slash.separated.date.partitioning=true`
## Logs and Stack Trace
No failure at write time -- the layouts simply diverge, and the Avro-written one does not read
back correctly.
Contributor guide
No contributing guide indexed for this repository
Research direction
Read HoodieWriterUtils.validateTableConfig and compare it with HoodieCatalogTable#extraTableConfig; trace the df.write.format("hudi") and HoodieStreamer entry points. Reproduce the write with both configs enabled, then verify every writer rejects the combination, including non-SQL paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100