[BUG] PartitionPathParser truncates slash-separated partition values for non-time-typed columns (2026/01/05 reads back as 2026)
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
## Bug Description
**What happened:**
`PartitionPathParser#getPartitionValues` splits the partition path on `/` and consumes exactly one
segment per partition field, unless the field is time-based:
```java
String[] parts = partitionPath.split("/");
...
if (isTimeBasedType(fieldSchema.getType())) {
...
int numDateDirs = parts.length - partitionFields.length + 1;
partitionValues[i] = inferDateValue(partitionPath, parts, pathSegment, numDateDirs, fieldSchema);
pathSegment += numDateDirs;
} else {
String segment = parts[pathSegment];
String[] segmentParts = segment.split(EQUALS_SIGN);
partitionValues[i] = parseValue(segmentParts[segmentParts.length - 1], fieldSchema);
pathSegment++;
}
```
(`PartitionPathParser.java:55-78`)
The multi-directory case is handled only for `DATE`, `TIMESTAMP` and `TIME`
(`isTimeBasedType`, `PartitionPathParser.java:164-166`). Under
`hoodie.datasource.write.slash.separated.date.partitioning=true` a partition column declared
`STRING` — which is a perfectly ordinary way to hold a `yyyy-MM-dd` value, and what the feature's
own test table uses — produces the directory `2026/01/05`, so `parts` is
`["2026", "01", "05"]` while `partitionFields.length` is 1. The `else` branch takes `parts[0]` and
the partition value is read back as **`"2026"`**. The remaining two segments are silently dropped.
This is not the Spark read path: `HoodieSparkUtils#doParsePartitionColumnValues` handles the slash
layout, which is why the datasource tests pass. `PartitionPathParser` is the engine-agnostic
parser, reached from `HoodieFileGroupReader` (`HoodieFileGroupReader.java:284-286`) when a
bootstrap table merges skeleton and data files. The wrong value is then handed to
`convertValueToEngineType` and materialized into the returned records.
So: a bootstrap table with slash-separated date partitioning and a STRING partition column returns
`2026` in the partition column for every row, rather than `2026-01-05`.
Calling `getPartitionFieldVals` directly against a schema with two string fields and one date field
shows the behavior:
| partition path | partition fields | result |
|---|---|---|
| `2026/01/05` | `[string_field]` | `[2026]` — truncated |
| `2026/01/05` | `[date_field]` | `[2026-01-05]` — the time-based branch handles it |
| `2026-01-05` | `[string_field]` | `[2026-01-05]` — unaffected without slash partitioning |
| `2026/01` | `[string_field]` | `[2026]` — truncated |
| `2026/01/05/us` | `[string_field, other_field]` | `[2026, 01]` — shifted, `us` dropped |
The last row is the same `else` branch failing a second way: consuming one segment per field means
a slash-partitioned field mis-aligns every field after it, so the values are not merely truncated
but land on the wrong columns. That case is a subset of #19666, which tracks multi-field slash
partitioning being broken generally; this issue is about the single-field truncation.
**What you expected:**
`PartitionPathParser` should reconstruct `2026-01-05` from `2026/01/05` for a non-time-typed
partition column when the table config has slash separated date partitioning enabled, matching what
`HoodieSparkUtils#doParsePartitionColumnValues` already does on the Spark side.
**Steps to reproduce:**
1. Create a bootstrap table with one `STRING` partition column and
`hoodie.datasource.write.slash.separated.date.partitioning=true`.
2. Write a row with partition value `2026-01-05`; the directory `/2026/01/05/` is created.
3. Read the table back through the file group reader's bootstrap merge path and inspect the
partition column: it holds `2026`.
**Suggested fix:**
Make the non-time-typed branch slash-aware rather than special-casing dates only: when the table
config enables slash separated date partitioning, consume `parts.length - partitionFields.length + 1`
segments for the field the same way the time-based branch already does, and rejoin them with `-`.
The config is not currently threaded into `PartitionPathParser`, so it would have to be passed in
alongside the schema.
Raised during review of #19648.
## Environment
**Hudi version:** master (1.3.0-SNAPSHOT)
**Query engine:** engine-agnostic (`HoodieFileGroupReader`); reproduced against Spark, but the
parser is shared
**Relevant configs:** `hoodie.datasource.write.slash.separated.date.partitioning=true` with a
`STRING` (or any non-`DATE`/`TIMESTAMP`/`TIME`) partition column
## Logs and Stack Trace
No exception: the partition value is silently truncated to the first path segment.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.