"Ambiguous" column name with dot
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Apache Iceberg version
1.9.2 (latest release)
### Query engine
Spark
### Please describe the bug 🐞
Hello Iceberg team,
When running the following query in Spark SQL:
```
CREATE TABLE local.db.errorTable
USING ICEBERG
AS SELECT
1 AS `A.B`,
FROM_JSON('{"B": 2}', 'B INT') AS `A`;
```
we encounter the following error during table creation:
```
25/07/09 21:54:46 ERROR SparkSQLDriver: Failed in [CREATE TABLE local.db.errorTable
USING ICEBERG
AS SELECT
1 AS `A.B`,
FROM_JSON('{"B": 2}', 'B INT') AS `A`]
org.apache.iceberg.exceptions.ValidationException: Invalid schema: multiple fields for name A.B: 0 and 2
at org.apache.iceberg.exceptions.ValidationException.check(ValidationException.java:49)
at org.apache.iceberg.types.IndexByName.addField(IndexByName.java:200)
at org.apache.iceberg.types.IndexByName.field(IndexByName.java:161)
```
This seems to happen because the schema validation treats a top-level column name containing a dot (``` `A.B` ```) as conflicting with a nested column path (``` `A` .`B` ```).
However, many SQL engines support such column names in accordance with the SQL-92 specification ([reference](https://db.apache.org/derby/docs/10.1/ref/crefsqlj1003454.html)).
From the stack trace, it appears the issue occurs in [IndexByName.addField](https://github.com/apache/iceberg/blob/efbfb7ef9addeb33e72208c927936e50b92d3357/api/src/main/java/org/apache/iceberg/types/IndexByName.java#L189-L196), which validates column names and detects collisions.
### Expected behavior:
The table should be created successfully, even if a top-level column name contains a dot, as long as it is distinct from nested field paths in the logical schema.
Allowing this case would align Iceberg’s behavior with SQL-92 and other SQL engine implementations.
### Environment:
Iceberg version: `1.9.2`
Thank you for your time and for maintaining this great project!
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
Contributor guide
Research direction
Start by reproducing the Spark SQL CREATE TABLE query, then inspect api/src/main/java/org/apache/iceberg/types/IndexByName.java, especially addField and the linked validation path. Confirm that a top-level A.B and nested A.B are treated as distinct logical fields, and verify that the table is created successfully without the multiple-fields validation error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 65/100