Cannot set `write.data.path` to a subdirectory under the current table location OPTIMIZED_SIBLING_CHECK=true
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 522
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 137
Description
### Describe the bug
Currently, when updating write.data.path of the table to a subdir under the table location, it will fail the location overlap check. This seems to be a regression since https://github.com/apache/polaris/pull/1686/changes#diff-1e0bdc1f194df7cceed5d4b2ddad483199b80d090a1c682f92ecf5fec5f83b2f
### To Reproduce
For example
```
spark-sql> ALTER TABLE tb1 SET TBLPROPERTIES (
'write.data.path' = '/alternative_data'
);
org.apache.iceberg.exceptions.ForbiddenException: Forbidden: Unable to create table at location 's3://' because it conflicts with existing table or namespace at location 's3://`
```
### Actual Behavior
_No response_
### Expected Behavior
The `validateNoLocationOverlap` should only prevent updating table location to be the same or child of sibling table's locations, not itself. Setting `write.data.path` to `/alternative` instead of the default `/data` should be a valid use case
### Additional context
The issue exists in both the active `validateNoLocationOverlap` path and the `OPTIMIZED_SIBLING_CHECK` path. The later have some other issues as described here: https://github.com/apache/polaris/issues/3378
### System information
_No response_
Contributor guide
Research direction
Trace the validateNoLocationOverlap path and the OPTIMIZED_SIBLING_CHECK path described in the issue, then compare them with the regression discussed in pull request 1686 and the related concerns in issue 3378. Reproduce the ALTER TABLE example and verify that changing write.data.path to a child directory of the same table location succeeds while sibling-table overlaps remain rejected.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100