apache / apache/iceberg

Table with no current snapshot throws on time travel filter with different schema

Open
#17,734 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Apache Iceberg version

1.11.0 (latest release)

### Query engine

None

### Please describe the bug 🐞

https://github.com/apache/iceberg/pull/13301 fixed https://github.com/apache/iceberg/issues/11162, but not for the case where the table currently has no snapshot.

For example, consider a WAP-style workflow where a table has snapshots potentially with different schemas, but does not have a snapshot on its main branch yet. If those snapshots with non-main schemas are read via time travel, planning a filter on a column that existed in the snapshot fails with e.g. (see https://github.com/apache/iceberg/pull/17735 for the failing test here)

```
Cannot find field 'data' in struct: struct<1: id: required long, 3: part: required string>
```

This is the same problem as in the above PR / issue, except the fix PR included a `currentSnapshot() == null` special case that causes the issue to remain when the table has no snapshots.

### Willingness to contribute

- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time

Contributor guide

Open the contributing guide

Research direction

Start by reading issue 11162 and pull request 13301, then inspect the failing test described in pull request 17735. Reproduce the time-travel filter failure for a table with no current snapshot and snapshots using different schemas. Done means the filter can resolve the column from the time-travel snapshot without the reported missing-field error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.