apache / apache/iceberg

Support preserving field IDs when creating a table from an existing schema

Open
#17,614 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

**Problem**

`TableMetadata.newTableMetadata()` unconditionally reassigns fresh, sequential field IDs via `TypeUtil.assignFreshIds(INITIAL_SCHEMA_ID, schema, lastColumnId::incrementAndGet)` ([TableMetadata.java#L121](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/TableMetadata.java#L121)), regardless of whatever field IDs were already set on the `Schema` object passed in to `Catalog.createTable`/`TableBuilder.create()`.

This means there is currently no way to create a table via the public `Catalog`/`TableBuilder` API while preserving specific, pre-existing field IDs on the schema — the IDs are always silently overwritten with a fresh sequential assignment starting from `INITIAL_SCHEMA_ID`.

**Request**

A supported, documented way to create/register a table while preserving an intended schema's field IDs exactly, instead of always reassigning fresh ones. Possible shapes:
- A `TableBuilder` option, e.g. `.withSchema(schema, /* preserveFieldIds= */ true)`.
- A distinct API method/contract for creating a table entry against a schema whose field IDs are already meaningful, as opposed to always treating the input as a brand-new schema.
- At minimum, explicit documentation on `Catalog.createTable` / `TableBuilder.create()` stating that field IDs are never preserved from the input schema.

Happy to discuss further or contribute a PR if there's agreement on the right shape for this.

Contributor guide

Open the contributing guide

Research direction

Start with TableMetadata.java around the cited newTableMetadata() call and trace how Catalog.createTable and TableBuilder.create() pass the schema to TypeUtil.assignFreshIds. Review the requested API alternatives and existing contracts before choosing a supported shape. Done means the chosen contract preserves intended field IDs and is documented, with behavior verified through the relevant creation path.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.