FilOzone / FilOzone/dealbot

Define cleanup policy for blocked providers and terminated datasets

Open
#462 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
3
Forks
9
Avg merge
4d 14h
Merged PRs (30d)
6

Description

## Context

Follow-up from PR #346 and issue #281.

- PR: https://github.com/FilOzone/dealbot/pull/346
- Parent issue: https://github.com/FilOzone/dealbot/issues/281

PR #346 adds the basic per-SP `piece_cleanup` job to bound storage growth by deleting old dealbot-owned pieces. During review, we identified two lifecycle policy questions that should be handled separately so the cleanup PR stays focused.

## Questions to resolve

### 1. Blocked providers

Provider blocklists currently stop scheduled data-storage/retrieval/dataset-creation checks for providers we do not want to test.

Cleanup may need different behavior: if an SP is blocked but still has old dealbot data, cleanup/reconciliation may still be useful so dealbot reduces storage load and keeps local DB state accurate.

Decide whether `piece_cleanup` should:

- ignore provider blocklists and continue cleanup,
- respect provider blocklists and skip cleanup,
- or use a separate cleanup-specific allow/block policy.

Initial recommendation: cleanup should probably still run for blocked providers, because blocking new checks should not prevent cleanup or DB reconciliation.

### 2. Terminated or non-live datasets

Dataset/service termination does not necessarily mean there is nothing to clean up. Depending on FWSS/filecoin-pay lifecycle state, a dataset may be unavailable for new additions while piece removal, settlement, or local DB reconciliation still matters.

Decide how dealbot should handle cleanup candidates from:

- live datasets where pieces can be actively removed,
- terminated/non-live datasets where removals may still be allowed,
- fully finalized/deleted datasets where local DB rows may need to be marked cleaned up without attempting provider deletion.

Initial recommendation: do not blindly filter non-live datasets out of quota/reconciliation until we define the lifecycle states and expected behavior.

## Done criteria

- Document the expected `piece_cleanup` behavior for blocked providers.
- Document how cleanup/reconciliation should treat live, terminated, and fully finalized/deleted datasets.
- Add implementation changes if needed.
- Add tests covering the selected policies.
- Link the resulting behavior from the jobs/runbook docs if operator action is needed.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.