apache / apache/fluss

[flink] Add reliable cleanup result summaries to remove_orphan_files

Open
#4,318 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Description

Users need to distinguish planned cleanup from actual deletion and determine whether a cleanup run covered all requested targets. Job completion alone does not provide this information, especially when metadata is unavailable or some targets cannot be scanned.

Separate planned and actual cleanup counters, account for incomplete scope coverage, and emit bounded, versioned result logs with a run ID and per-subtask summaries. Validate counter consistency and ensure dry-run results report no actual deletion.

### Willingness to contribute

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Locate the remove_orphan_files entry point and trace how cleanup scope, deletion outcomes, dry-run handling, and existing logs are represented. Define tests for planned versus actual counters, incomplete coverage, bounded versioned summaries with run IDs and per-subtask data, counter consistency, and zero actual deletions in dry runs; done when those behaviors are validated.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.