Task handling is incomplete
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 522
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 137
Description
### Describe the bug
Polaris uses some asynchronously executed tasks to run operations for table and manifest file cleanup. Those tasks are potentially executed in a separate thread in the same JVM. There is however no guarantee that those tasks will eventually run for multiple reasons:
* Tasks (e.g. via `org.apache.polaris.service.catalog.BasePolarisCatalog#dropTable`) are triggered _after_ the fact.
* Although tasks are persisted, there is no mechanism to pick up tasks that did not start or did not finish ("long lasting" failures, JVM terminates).
Overall this means that for example a "drop table with purge" returns a successful result to the user, the actual purge may never ever happen.
### To Reproduce
_No response_
### Actual Behavior
_No response_
### Expected Behavior
_No response_
### Additional context
_No response_
### System information
_No response_
Contributor guide
Research direction
Start at org.apache.polaris.service.catalog.BasePolarisCatalog#dropTable and trace how cleanup tasks are triggered and persisted. Review how tasks that never start or do not finish are handled; done should ensure operations such as table drops with purge cannot report success while cleanup is permanently abandoned.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100