[Bug]: IcebergTableMaintainer orphan file cleanup can corrupt other tables sharing the same location
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 395
- Avg merge
- 4d 10h
- Merged PRs (30d)
- 33
Description
### What happened?
## Bug description
Amoro's `IcebergTableMaintainer.cleanOrphanFiles()` can delete files belonging to *other* Iceberg tables when two tables share the same `LOCATION`.
It computes reachable files from only the target table's metadata, then recursively lists the whole location and deletes everything else — including a second table's data, manifest, and metadata files under the same directory.
This can be triggered inadvertently in practice: for example, a table is created by copying `SHOW CREATE TABLE` of another table but forgetting to change `LOCATION`. We hit this in production (irreversible data loss) and found 100+ shared-location pairs in our environment.
## Proposed fix
Add a default-on guard in `IcebergTableMaintainer` before orphan cleanup:
1. List `metadata/*.metadata.json` (+ `.metadata.json.gz`) under `table.location()/metadata`.
2. For each file not in the table's own version set, read its `table-uuid`:
- Different uuid → another table shares the location → **skip cleanup** (default).
- Unreadable (legacy/compressed/corrupt) → treat as suspicious → skip (fail-safe).
- Same uuid → older version of this table → ignore.
3. Only proceed when no foreign uuid is found.
Add table property `clean-orphan-file.ignore-location-conflict` (default `false`) to opt out when a shared location is intentional.
### Affects Versions
all
### What table formats are you seeing the problem on?
Iceberg
### What engines are you seeing the problem on?
AMS
### How to reproduce
1. Create Iceberg table A at `hdfs://ns/tmp/shared`.
2. Create Iceberg table B at the **same** location `hdfs://ns/tmp/shared`.
3. Configure orphan file cleanup and trigger it on A:
- Set catalog-level table property `clean-orphan-file.enabled=true` to enable periodic cleanup.
- Set `clean-orphan-file.min-existing-time-minutes=2` so orphan files are eligible for deletion within minutes (instead of the default 2 days), for quick observation.
4. Restart AMS so the configuration takes effect, and let the scheduled orphan file cleanup task run.
5. B's files under the shared directory are deleted → B becomes corrupted/unloadable.
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's Code of Conduct
Contributor guide
Research direction
Start with IcebergTableMaintainer.cleanOrphanFiles() and trace how reachable files and the recursive location listing are computed. Reproduce the shared-location case with two Iceberg tables, then verify cleanup skips when metadata reveals a foreign table UUID, while the default-off conflict property permits intentional sharing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100