`TableCreation::format_version` is ignored when creating a table using the Iceberg REST catalog interface
- Dominant language
- Rust
- Stars
- 1.4k
- Forks
- 567
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 93
Description
### Apache Iceberg Rust version
None
### Describe the bug
When using the Rest catalog, the `TableCreation::format_version` is silently ignored and never used. The result is you can't set a table to be V3 when using `format_version` and a REST catalog.
# Steps to reproduce
* Spin up a Rest catalog (say Polaris), and use that.
* Make a new table.
* Set it to use `format_version(FormatVersion::V3)`
* Then add a V3 column, and try to use the table.
* Outcome: the table is in V2 format (not v3).
The table building code would be something like this:
```rust
let table = TableCreation::builder()
.name(table_name)
.schema(table_schema)
.format_version(FormatVersion::V3) // <-- this call here is ignored
.build();
```
# Context
In the Rest `catalog.rs` here the tables `.properties` are passed, and `format_version` is never used: https://github.com/apache/iceberg-rust/blob/main/crates/catalog/rest/src/catalog.rs#L794-L815
A fix could be to either add it to the properties here, or add it to the properties upstream.
# Workaround
Set the table properties to use V3 by hand like so:
```rust
let table = TableCreation::builder()
.name(table_name)
.schema(table_schema)
.partition_spec(table_partition_spec)
.properties(HashMap::from([
// Format v3 is needed as we are using nanosecond times.
(
TableProperties::PROPERTY_FORMAT_VERSION.to_string(),
"3".to_string(),
),
]))
.build();
```
### To Reproduce
_No response_
### Expected behavior
* The format version set by `format_version` should be used, when it is set on it's own.
I'm not 100% sure what should happen when setting the version by both `format_version` and `properties`, or if we even care about having precise behaviour there.
### Willingness to contribute
I can contribute a fix for this bug independently
Contributor guide
Research direction
Start in crates/catalog/rest/src/catalog.rs around lines 794-815 and trace how TableCreation properties and format_version become the REST table-creation request. Confirm the V3 reproduction, then ensure a format_version set on its own reaches the request and add regression coverage for the expected version behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, databases
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100