Aliyun/Dell: Classify missing object reads as NotFoundException
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Apache Iceberg version
1.10.1 (latest release), main (development)
### Query engine
Other
### Please describe the bug 🐞
`BaseMetastoreTableOperations.refreshFromMetadataLocation` retries metadata reads and stops retrying when loading fails with Iceberg `NotFoundException`. Some object-storage read paths do not classify permanent missing-object failures as `NotFoundException`, so a stale catalog metadata location can exhaust the generic metadata refresh retry budget instead of failing fast.
This was observed through Apache Gravitino with Aliyun OSS metadata locations: https://github.com/apache/gravitino/issues/11039
Current behavior:
- AWS S3, Azure ADLS, GCS, and Hadoop input streams translate missing files or objects to `NotFoundException`.
- Aliyun OSS `exists()` returns `false` for `NO_SUCH_KEY` / `NO_SUCH_BUCKET`, but `OSSInputStream.openStream` calls `client.getObject(...)` directly and can propagate Aliyun SDK `OSSException`.
- Dell ECS `exists()` returns `false` for HTTP 404, but `EcsSeekableInputStream` calls `client.readObjectStream(...)` directly and can propagate Dell SDK `S3Exception`.
Expected behavior:
Aliyun OSS and Dell ECS stream-read paths should classify permanent missing-object failures as Iceberg `NotFoundException`, matching other FileIO implementations and allowing metadata refresh to stop retrying stale metadata locations.
### Willingness to contribute
- [X] I can contribute a fix for this bug independently
Contributor guide
Assessment
This issue has not been assessed yet.