apache / apache/hudi

[BUG] PartitionPathParser truncates slash-separated partition values for non-time-typed columns (2026/01/05 reads back as 2026)

Open
#19,705 0 comments 0 reactions 1 assignee Claimed by @SEPURI-SAI-KRISHNA View on GitHub
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

## Bug Description

**What happened:**

`PartitionPathParser#getPartitionValues` splits the partition path on `/` and consumes exactly one
segment per partition field, unless the field is time-based:

```java
String[] parts = partitionPath.split("/");
...
if (isTimeBasedType(fieldSchema.getType())) {
...
int numDateDirs = parts.length - partitionFields.length + 1;
partitionValues[i] = inferDateValue(partitionPath, parts, pathSegment, numDateDirs, fieldSchema);
pathSegment += numDateDirs;
} else {
String segment = parts[pathSegment];
String[] segmentParts = segment.split(EQUALS_SIGN);
partitionValues[i] = parseValue(segmentParts[segmentParts.length - 1], fieldSchema);
pathSegment++;
}
```
(`PartitionPathParser.java:55-78`)

The multi-directory case is handled only for `DATE`, `TIMESTAMP` and `TIME`
(`isTimeBasedType`, `PartitionPathParser.java:164-166`). Under
`hoodie.datasource.write.slash.separated.date.partitioning=true` a partition column declared
`STRING` — which is a perfectly ordinary way to hold a `yyyy-MM-dd` value, and what the feature's
own test table uses — produces the directory `2026/01/05`, so `parts` is
`["2026", "01", "05"]` while `partitionFields.length` is 1. The `else` branch takes `parts[0]` and
the partition value is read back as **`"2026"`**. The remaining two segments are silently dropped.

This is not the Spark read path: `HoodieSparkUtils#doParsePartitionColumnValues` handles the slash
layout, which is why the datasource tests pass. `PartitionPathParser` is the engine-agnostic
parser, reached from `HoodieFileGroupReader` (`HoodieFileGroupReader.java:284-286`) when a
bootstrap table merges skeleton and data files. The wrong value is then handed to
`convertValueToEngineType` and materialized into the returned records.

So: a bootstrap table with slash-separated date partitioning and a STRING partition column returns
`2026` in the partition column for every row, rather than `2026-01-05`.

Calling `getPartitionFieldVals` directly against a schema with two string fields and one date field
shows the behavior:

| partition path | partition fields | result |
|---|---|---|
| `2026/01/05` | `[string_field]` | `[2026]` — truncated |
| `2026/01/05` | `[date_field]` | `[2026-01-05]` — the time-based branch handles it |
| `2026-01-05` | `[string_field]` | `[2026-01-05]` — unaffected without slash partitioning |
| `2026/01` | `[string_field]` | `[2026]` — truncated |
| `2026/01/05/us` | `[string_field, other_field]` | `[2026, 01]` — shifted, `us` dropped |

The last row is the same `else` branch failing a second way: consuming one segment per field means
a slash-partitioned field mis-aligns every field after it, so the values are not merely truncated
but land on the wrong columns. That case is a subset of #19666, which tracks multi-field slash
partitioning being broken generally; this issue is about the single-field truncation.

**What you expected:**

`PartitionPathParser` should reconstruct `2026-01-05` from `2026/01/05` for a non-time-typed
partition column when the table config has slash separated date partitioning enabled, matching what
`HoodieSparkUtils#doParsePartitionColumnValues` already does on the Spark side.

**Steps to reproduce:**
1. Create a bootstrap table with one `STRING` partition column and
`hoodie.datasource.write.slash.separated.date.partitioning=true`.
2. Write a row with partition value `2026-01-05`; the directory `/2026/01/05/` is created.
3. Read the table back through the file group reader's bootstrap merge path and inspect the
partition column: it holds `2026`.

**Suggested fix:**

Make the non-time-typed branch slash-aware rather than special-casing dates only: when the table
config enables slash separated date partitioning, consume `parts.length - partitionFields.length + 1`
segments for the field the same way the time-based branch already does, and rejoin them with `-`.
The config is not currently threaded into `PartitionPathParser`, so it would have to be passed in
alongside the schema.

Raised during review of #19648.

## Environment

**Hudi version:** master (1.3.0-SNAPSHOT)
**Query engine:** engine-agnostic (`HoodieFileGroupReader`); reproduced against Spark, but the
parser is shared
**Relevant configs:** `hoodie.datasource.write.slash.separated.date.partitioning=true` with a
`STRING` (or any non-`DATE`/`TIMESTAMP`/`TIME`) partition column

## Logs and Stack Trace

No exception: the partition value is silently truncated to the first path segment.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.