apache / apache/iceberg-rust

MetadataLocation parser rejects Java HadoopCatalog's v<n>.metadata.json naming

Open Beginner friendly
#2,533 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

## problem

`MetadataLocation::from_str` only accepts the `-.metadata.json` format. java's `HadoopTableOperations` writes metadata as `v1.metadata.json`, `v2.metadata.json`, etc. when iceberg-rust tries to commit on top of a table created by java's HadoopCatalog, parsing fails:

```
Error: Invalid metadata file name: v1.metadata.json
```

this blocks any commit operation on java HadoopCatalog-created tables.

## reproduction

1. create a table using java iceberg with HadoopCatalog (produces `v1.metadata.json`)
2. load the table in iceberg-rust via any catalog that stores the full metadata path
3. attempt a `Transaction::commit` — fails at `MetadataLocation::from_str`

## relevant code

`crates/iceberg/src/catalog/metadata_location.rs` — `parse_file_name` splits on `-` which doesn't exist in the java format.

## expected behavior

`parse_file_name` should accept both formats:
- rust-native: `-.metadata.json`
- java hadoop: `v.metadata.json`

the version number is the only thing needed from the parse — it drives `with_next_version()`.

## notes

- the iceberg spec does not prescribe a metadata file naming convention
- java's HadoopCatalog is widely used in production (hadoop/HDFS deployments)
- this is read+commit interop, not a full HadoopCatalog implementation

Contributor guide

Open the contributing guide

Research direction

Start in crates/iceberg/src/catalog/metadata_location.rs, especially parse_file_name and MetadataLocation::from_str. Trace how the parsed version is used by with_next_version(), then verify that both Rust-native and Java HadoopCatalog metadata names parse and that Transaction::commit works for the reproduced scenario.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.