apache / apache/iceberg

Double slash (//) in table location path

Open
#15,908 17 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Java
Stars
9.2k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
132

Description

### Apache Iceberg version

1.10.1 (latest release)

### Query engine

Spark

### Please describe the bug 🐞

### Problem
When creating Iceberg tables through Spark with S3 location, the table location contains double slashes (`//`) in the path. This behavior occurs regardless of whether the table is partitioned or not. The double slash in the path causes the `OPTIMIZE` operation to behave incorrectly and sometimes delete files, leading to table corruption.

### Example
```sql
CREATE TABLE test.test.test (
test_rk integer
)
WITH (
compression_codec = 'ZSTD',
format = 'PARQUET',
format_version = 2,
location = 's3a://test//test_8262bea6c787' -- ⚠️ Double slash after 'test/'
)
```
### Example 2
```sql
CREATE TABLE test.test.test (
test_rk decimal(21, 0),
test real,
test_l varchar,
test_stat real,
test_id integer
)
WITH (
compression_codec = 'ZSTD',
format = 'PARQUET',
format_version = 2,
location = 's3a://test//test',
partitioning = ARRAY['test_id']
)

```

Component | Version
-- | --
iceberg-spark-runtime | 3.5_2.12-1.10.1
iceberg-aws-bundle | 1.10.1
Spark | 3.5.6
Filesystem | S3A

simple spark query: `df.format('iceberg').mode('overwrite').saveAsTable("test.test.test")`

### Willingness to contribute

- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [x] I cannot contribute a fix for this bug at this time

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the table creation with the shown Spark saveAsTable call and S3A locations, then trace how Iceberg records and normalizes the table location before OPTIMIZE. Done means the resulting location does not contain the unintended double slash and OPTIMIZE no longer deletes files or corrupts the table.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, hadoop, java
Domain
data-engineering, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.