FilOzone / FilOzone/dealbot

Dealbot accumulates many active data sets per provider (calib: 196 on provider 23) — no reuse/cleanup

Open
#605 12 comments 0 reactions 1 assignee Claimed by @beck-8 View on GitHub
Dominant language
TypeScript
Stars
3
Forks
9
Avg merge
4d 14h
Merged PRs (30d)
6

Description

## Problem

On calibration, dealbot's wallet (`0x305025…dca0b`) holds a large and growing number of **active (un-terminated, `pdpEndEpoch=0n`) data sets per provider**, instead of reusing or cleaning them up:

| provider | active dealbot data sets |
|---|---|
| 23 (🚀, `0x6c8E…d796`) | **196** |
| 4, 6 | 25 |
| 2, 9 | 24 |
| 5, 8, 15, 22, 27 | 15 |
| 26 | 7 |
| 24 (sp-playground) | 0 (terminates all → contrast) |

Provider 23's 196: **182 share identical metadata** `{"source":"dealbot","withIPFSIndexing":""}` (data set ids 13115–13751), 14 are unique `dealbotDS:N` indexed slots; **all have pieces, zero terminated**. (Source: Foc-observer `fwss_data_set_created` LEFT JOIN `fwss_service_terminated`.)

## Why it matters

`StorageContext.resolveByProviderId` (synapse-sdk) scans **every** active data set for the provider on each deal to metadata-select one. So this count *is* the per-resolve RPC fan-out width — provider 23 alone forces ~132 `eth_call`s per resolve and it grows daily. See the RPC-burst tracking issue. (Also wastes storage/payment rails.)

## Open question (to resolve)

Are the 182 identical-metadata sets a **reuse failure** (resolveByProviderId should return the existing metadata-matching set, but dealbot creates a new one anyway) or **intended fresh-create** by `data_set_creation` without cleanup? Provider 24 terminates everything; provider 23 terminates nothing — so cleanup is at least inconsistent.

## Next steps

- Determine the create-vs-reuse path for the base-metadata sets (instrument `createContext`/`createDataSetWithPiece`).
- Decide policy: reuse the existing match, or terminate old test data sets (and why provider 23 never terminates while 24 does).
- Sweep the existing accumulated sets on calib.

## Done criteria

- Active dealbot data-set count per provider stays bounded on calib.
- Root of the 182-set accumulation identified and fixed (reuse or cleanup).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.