apache / apache/iceberg

"Ambiguous" column name with dot

Open
#13,807 4 comments 0 reactions 0 assignees View on GitHub
bug stale
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Apache Iceberg version

1.9.2 (latest release)

### Query engine

Spark

### Please describe the bug 🐞

Hello Iceberg team,

When running the following query in Spark SQL:
```
CREATE TABLE local.db.errorTable
USING ICEBERG
AS SELECT
1 AS `A.B`,
FROM_JSON('{"B": 2}', 'B INT') AS `A`;
```

we encounter the following error during table creation:
```
25/07/09 21:54:46 ERROR SparkSQLDriver: Failed in [CREATE TABLE local.db.errorTable
USING ICEBERG
AS SELECT
1 AS `A.B`,
FROM_JSON('{"B": 2}', 'B INT') AS `A`]
org.apache.iceberg.exceptions.ValidationException: Invalid schema: multiple fields for name A.B: 0 and 2
at org.apache.iceberg.exceptions.ValidationException.check(ValidationException.java:49)
at org.apache.iceberg.types.IndexByName.addField(IndexByName.java:200)
at org.apache.iceberg.types.IndexByName.field(IndexByName.java:161)
```

This seems to happen because the schema validation treats a top-level column name containing a dot (``` `A.B` ```) as conflicting with a nested column path (``` `A` .`B` ```).

However, many SQL engines support such column names in accordance with the SQL-92 specification ([reference](https://db.apache.org/derby/docs/10.1/ref/crefsqlj1003454.html)).

From the stack trace, it appears the issue occurs in [IndexByName.addField](https://github.com/apache/iceberg/blob/efbfb7ef9addeb33e72208c927936e50b92d3357/api/src/main/java/org/apache/iceberg/types/IndexByName.java#L189-L196), which validates column names and detects collisions.

### Expected behavior:
The table should be created successfully, even if a top-level column name contains a dot, as long as it is distinct from nested field paths in the logical schema.

Allowing this case would align Iceberg’s behavior with SQL-92 and other SQL engine implementations.

### Environment:

Iceberg version: `1.9.2`

Thank you for your time and for maintaining this great project!

### Willingness to contribute

- [ ] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Spark SQL CREATE TABLE query, then inspect api/src/main/java/org/apache/iceberg/types/IndexByName.java, especially addField and the linked validation path. Confirm that a top-level A.B and nested A.B are treated as distinct logical fields, and verify that the table is created successfully without the multiple-fields validation error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.