apache / apache/iceberg

DELETE operations don't apply custom snapshot properties from session config

Open Beginner friendly
#15,060 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Apache Iceberg version

main (development)

### Query engine

Spark

### Please describe the bug 🐞

#### Description
Custom snapshot properties set via Spark session config (e.g., `spark.sql.iceberg.snapshot-property.custom-key`) are not applied to DELETE operations, while they work correctly for INSERT, UPDATE, and MERGE operations.

#### Expected Behavior
DELETE should respect custom snapshot properties from session config, consistent with other DML operations.

#### Actual Behavior
DELETE operations silently ignore custom snapshot properties.

#### Steps to Reproduce
```java
spark.conf().set("spark.sql.iceberg.snapshot-property.my-key", "my-value");

// INSERT works - snapshot has my-key ✅
sql("INSERT INTO table VALUES (1, 'a')");

// DELETE doesn't work - snapshot missing my-key ❌
sql("DELETE FROM table WHERE id = 1");
```

#### Root Cause

In SparkTable.deleteFromRowFilter(), the code doesn't call SparkWriteConf.extraSnapshotMetadata() to apply session properties, unlike insert/update paths.

#### Impact

Users relying on custom snapshot properties for tracking (e.g., audit trails, job metadata) will have incomplete information for DELETE operations.

#### Related

- Feature introduced in #14545
- Affects Spark 4.0 and 4.1

### Willingness to contribute

- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time

Contributor guide

Open the contributing guide

Research direction

Start in SparkTable.deleteFromRowFilter() and compare its handling with the insert, update, and merge paths. Check how SparkWriteConf.extraSnapshotMetadata() is used, then run the provided Spark session configuration and DELETE reproduction. Done means the DELETE snapshot contains the configured custom property, matching other DML operations.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
database
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.