apache / apache/polaris

Cannot set `write.data.path` to a subdirectory under the current table location OPTIMIZED_SIBLING_CHECK=true

Open
#3,379 2 comments 0 reactions 0 assignees View on GitHub
bug stale
Dominant language
Java
Stars
2.1k
Forks
522
Avg merge
1d 17h
Merged PRs (30d)
137

Description

### Describe the bug

Currently, when updating write.data.path of the table to a subdir under the table location, it will fail the location overlap check. This seems to be a regression since https://github.com/apache/polaris/pull/1686/changes#diff-1e0bdc1f194df7cceed5d4b2ddad483199b80d090a1c682f92ecf5fec5f83b2f

### To Reproduce

For example
```
spark-sql> ALTER TABLE tb1 SET TBLPROPERTIES (
'write.data.path' = '/alternative_data'
);

org.apache.iceberg.exceptions.ForbiddenException: Forbidden: Unable to create table at location 's3://' because it conflicts with existing table or namespace at location 's3://`
```

### Actual Behavior

_No response_

### Expected Behavior

The `validateNoLocationOverlap` should only prevent updating table location to be the same or child of sibling table's locations, not itself. Setting `write.data.path` to `/alternative` instead of the default `/data` should be a valid use case

### Additional context

The issue exists in both the active `validateNoLocationOverlap` path and the `OPTIMIZED_SIBLING_CHECK` path. The later have some other issues as described here: https://github.com/apache/polaris/issues/3378

### System information

_No response_

Contributor guide

Open the contributing guide

Research direction

Trace the validateNoLocationOverlap path and the OPTIMIZED_SIBLING_CHECK path described in the issue, then compare them with the regression discussed in pull request 1686 and the related concerns in issue 3378. Reproduce the ALTER TABLE example and verify that changing write.data.path to a child directory of the same table location succeeds while sibling-table overlaps remain rejected.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, spark, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.