apache / apache/iceberg

Aliyun/Dell: Classify missing object reads as NotFoundException

Open
#16,299 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Apache Iceberg version

1.10.1 (latest release), main (development)

### Query engine

Other

### Please describe the bug 🐞

`BaseMetastoreTableOperations.refreshFromMetadataLocation` retries metadata reads and stops retrying when loading fails with Iceberg `NotFoundException`. Some object-storage read paths do not classify permanent missing-object failures as `NotFoundException`, so a stale catalog metadata location can exhaust the generic metadata refresh retry budget instead of failing fast.

This was observed through Apache Gravitino with Aliyun OSS metadata locations: https://github.com/apache/gravitino/issues/11039

Current behavior:

- AWS S3, Azure ADLS, GCS, and Hadoop input streams translate missing files or objects to `NotFoundException`.
- Aliyun OSS `exists()` returns `false` for `NO_SUCH_KEY` / `NO_SUCH_BUCKET`, but `OSSInputStream.openStream` calls `client.getObject(...)` directly and can propagate Aliyun SDK `OSSException`.
- Dell ECS `exists()` returns `false` for HTTP 404, but `EcsSeekableInputStream` calls `client.readObjectStream(...)` directly and can propagate Dell SDK `S3Exception`.

Expected behavior:

Aliyun OSS and Dell ECS stream-read paths should classify permanent missing-object failures as Iceberg `NotFoundException`, matching other FileIO implementations and allowing metadata refresh to stop retrying stale metadata locations.

### Willingness to contribute

- [X] I can contribute a fix for this bug independently

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.