Dealbot accumulates many active data sets per provider (calib: 196 on provider 23) — no reuse/cleanup
- Dominant language
- TypeScript
- Stars
- 3
- Forks
- 9
- Avg merge
- 4d 14h
- Merged PRs (30d)
- 6
Description
## Problem
On calibration, dealbot's wallet (`0x305025…dca0b`) holds a large and growing number of **active (un-terminated, `pdpEndEpoch=0n`) data sets per provider**, instead of reusing or cleaning them up:
| provider | active dealbot data sets |
|---|---|
| 23 (🚀, `0x6c8E…d796`) | **196** |
| 4, 6 | 25 |
| 2, 9 | 24 |
| 5, 8, 15, 22, 27 | 15 |
| 26 | 7 |
| 24 (sp-playground) | 0 (terminates all → contrast) |
Provider 23's 196: **182 share identical metadata** `{"source":"dealbot","withIPFSIndexing":""}` (data set ids 13115–13751), 14 are unique `dealbotDS:N` indexed slots; **all have pieces, zero terminated**. (Source: Foc-observer `fwss_data_set_created` LEFT JOIN `fwss_service_terminated`.)
## Why it matters
`StorageContext.resolveByProviderId` (synapse-sdk) scans **every** active data set for the provider on each deal to metadata-select one. So this count *is* the per-resolve RPC fan-out width — provider 23 alone forces ~132 `eth_call`s per resolve and it grows daily. See the RPC-burst tracking issue. (Also wastes storage/payment rails.)
## Open question (to resolve)
Are the 182 identical-metadata sets a **reuse failure** (resolveByProviderId should return the existing metadata-matching set, but dealbot creates a new one anyway) or **intended fresh-create** by `data_set_creation` without cleanup? Provider 24 terminates everything; provider 23 terminates nothing — so cleanup is at least inconsistent.
## Next steps
- Determine the create-vs-reuse path for the base-metadata sets (instrument `createContext`/`createDataSetWithPiece`).
- Decide policy: reuse the existing match, or terminate old test data sets (and why provider 23 never terminates while 24 does).
- Sweep the existing accumulated sets on calib.
## Done criteria
- Active dealbot data-set count per provider stays bounded on calib.
- Root of the 182-set accumulation identified and fixed (reuse or cleanup).
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.