apache / apache/iceberg

Equality deletes ignored on executor cache hit with different query

Open
#15,039 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Apache Iceberg version

1.9.2

### Query engine

Spark

### Please describe the bug 🐞

## Repro 1
### Steps
- Create an unpartitioned table `table1` with the following schema:
```
key BIGINT NOT NULL,
inserted_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP(),
revenue DECIMAL(15, 3),
comment STRING
```

- Populate table with data including some equality deletes
- Set maxExecutors=1 to ensure all jobs go to the same executor
- Run the following queries:
```
SELECT
date_trunc('year', inserted_at) AS year,
COUNT(*) AS count
FROM table1
GROUP BY 1 ORDER BY 1 ASC;

SELECT
1,
COUNT(*) AS count
FROM table1
GROUP BY 1 ORDER BY 1 ASC;
```

### Observed behavior
- Results for the second query are incorrect. The incorrect result-set is consistent with equality deletes being ignored.

## Repro 2
- Same as repro 1 but switch the two queries:
```
SELECT
1,
COUNT(*) AS count
FROM table1
GROUP BY 1 ORDER BY 1 ASC;

SELECT
date_trunc('year', inserted_at) AS year,
COUNT(*) AS count
FROM table1
GROUP BY 1 ORDER BY 1 ASC;
```

### Observed behavior
- Results for the second query are incorrect. The incorrect result-set is consistent with equality deletes being ignored.

## Non-repros
- No repro if `spark.sql.iceberg.executor-cache.enabled` is set to `false`
- No repro if the same query is executed repeatedly

## Comments
I have low familiarity with the Iceberg codebase, but could it be because BaseDeleteLoader.getOrReadEqDeletes uses deleteFile.location() as a key regardless of projection? https://github.com/apache/iceberg/blob/7f81e1e93084e50fa3676c2e131722f66a26b385/data/src/main/java/org/apache/iceberg/data/BaseDeleteLoader.java#L121

Should projection somehow be part of the cache key there?

### Willingness to contribute

- [ ] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time

Contributor guide

Open the contributing guide

Research direction

Start at data/src/main/java/org/apache/iceberg/data/BaseDeleteLoader.java, especially getOrReadEqDeletes, and inspect how deleteFile.location() is used when executor caching is enabled. Reproduce both query orders with Spark and maxExecutors=1, then verify that the second query still applies equality deletes; compare with executor caching disabled and repeated identical queries.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.