apache / apache/hudi

[BUG] Row-writer INSERT OVERWRITE bypasses the pending-clustering check when meta fields are not populated

Open
#19,770 0 comments 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

## Bug Description

**What happened:**

On a table with `hoodie.populate.meta.fields=false`, a row-writer `INSERT OVERWRITE` in dynamic partition mode goes through while a clustering plan is pending on the very file groups it replaces. No `HoodieClusteringUpdateException`, the plan stays pending, and the partition's data is replaced underneath it. The same statement on a table with meta fields populated is rejected, as #18829 intended.

**To reproduce** (Spark SQL, COW):

```sql
set hoodie.spark.sql.insert.into.operation=bulk_insert;
set hoodie.datasource.overwrite.mode=dynamic;
create table t (id int, name string, price double, dt string) using hudi
tblproperties (primaryKey = 'id', 'hoodie.populate.meta.fields' = 'false') partitioned by (dt) location '/tmp/t';
insert into t values (1, 'a1', 10, '2021-07-18');
insert into t values (2, 'a2', 20, '2021-07-18');
insert into t values (3, 'a3', 30, '2021-07-19');
call run_clustering(table => 't', op => 'schedule', selected_partitions => 'dt=2021-07-18'); -- 1 pending plan
insert overwrite table t partition (dt) values (1, 'a1_new', 11, '2021-07-18'); -- succeeds; expected: rejected
select id, name, dt from t; -- (1, a1_new, 2021-07-18): row 2 is gone, plan still pending
```

Drop the `'hoodie.populate.meta.fields' = 'false'` property and the last `insert overwrite` throws `Not allowed to update the clustering file group ...`.

**Cause:**

`BaseDatasetBulkInsertCommitActionExecutor.rejectIfOverlappingPendingClustering` (added by #18829) resolves the partitions being replaced through `DatasetBulkInsertOverwriteCommitActionExecutor.resolveTargetPartitions`, whose dynamic arm reads `_hoodie_partition_path` off the prepared dataset:

```java
// DatasetBulkInsertOverwriteCommitActionExecutor.java:94-100
// Dynamic partition path: read the populated _hoodie_partition_path meta field. The base
// class invokes this hook after HoodieDatasetBulkInsertHelper.prepareForBulkInsert, so the
// field is guaranteed to be present and populated by the configured key generator.
return preparedRecords.select(HoodieRecord.PARTITION_PATH_METADATA_FIELD).distinct()...
```

That guarantee does not hold when meta fields are off: `HoodieDatasetBulkInsertHelper.prepareForBulkInsert` stubs every meta column as a null literal (`HoodieDatasetBulkInsertHelper.scala:145`, `metaFieldsStubs = metaFields.map(f => Alias(Literal.create(null, StringType), f.name)())`). `resolveTargetPartitions` therefore returns `[null]`, `getLatestFileSlices(null)` is asked of the file-system view (with the embedded timeline server the request is `slices/partition/latest/?partition` with an empty value), nothing comes back, `fileGroupsToBeReplaced` is empty and the check returns without consulting the update strategy. The write then proceeds, and `getPartitionToReplacedFileIds` records the real partition from the write statuses, so the replacement itself is committed correctly; only the guard is skipped.

The static arm is unaffected (it reads `STATIC_OVERWRITE_PARTITION_PATHS` from config), and the RDD path is unaffected (`SparkInsertOverwriteCommitActionExecutor` derives partitions from the records' keys).

A second, adjacent inconsistency in the same method: `execute()` picks the partitioner from `writeConfig.getBoolean(HoodieTableConfig.POPULATE_META_FIELDS)` (`BaseDatasetBulkInsertCommitActionExecutor.java:117`) while `prepareForBulkInsert` decides whether to stub the columns from `config.populateMetaFields()` (`HoodieDatasetBulkInsertHelper.scala:77`), which since #19205 is derived from `hoodie.meta.fields.mode` on v10 tables. #19378 listed this site under "guards bypassable by setting only the mode" and was closed; the two reads can still disagree.

**Expected behavior:**

The pending-clustering check should reject (or, with `SparkAllowUpdateStrategy`, defer) the overwrite regardless of whether meta fields are populated. When `_hoodie_partition_path` is not populated, `resolveTargetPartitions` needs another source for the partitions: the key generator output that `prepareForBulkInsert` already computes, or the distinct partition values of the incoming rows via the configured partition path field.

**Related:**

- #18830 (open) describes the general symptom this is a remaining case of; #18829 fixed it for tables with meta fields populated.
- #19378 (closed) touched the same raw-boolean read but not the partition-path resolution.

## Environment

Master `18ae8c349058` (the executor and helper code are unchanged at head), Spark 3.5 / Scala 2.12 profile, local filesystem, metadata table enabled, embedded timeline server on. Found while adding row-writer pending-clustering coverage on #19163; the new tests there cover only the populated-meta-fields case, which passes.

## Logs and Stack Trace

Timeline-server requests issued by the overwrite in the failing run (the first is the guard's lookup, the second the post-write replacement bookkeeping):

```
GET /v1/hoodie/view/slices/partition/latest/?partition&lastinstantts=20260827103228088&...
GET /v1/hoodie/view/slices/partition/latest/?partition=dt%3D2021-07-18&lastinstantts=20260827103228088&...
```

Probe output:

```
PENDING=1
NONOVERLAP RESULT=NO_EXCEPTION
OVERLAP RESULT=NO_EXCEPTION
ROWS=(1,a1_new,2021-07-18) (4,b1,2021-07-19)
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with BaseDatasetBulkInsertCommitActionExecutor.rejectIfOverlappingPendingClustering and DatasetBulkInsertOverwriteCommitActionExecutor.resolveTargetPartitions, then inspect HoodieDatasetBulkInsertHelper.scala:77 and :145. Run the Spark SQL reproduction with meta fields disabled and compare it with the existing row-writer clustering coverage from #19163. Done means dynamic INSERT OVERWRITE rejects or defers when its target file groups have a pending clustering plan, regardless of meta-field settings.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, scala, spark
Domain
data-engineering, databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
56/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.