filecoin-project / filecoin-project/filecoin-pin
waitForIpniProviderResults: surface per-CID outcomes for diagnostic visibility
- Dominant language
- TypeScript
- Stars
- 26
- Forks
- 21
- Avg merge
- 5d 18h
- Merged PRs (30d)
- 27
Description
## Problem
`waitForIpniProviderResults` returns `Promise` and on failure throws a single `Error` whose message references one CID (the last one checked). Consumers cannot tell, after the fact, which CIDs in `[rootCid, ...childBlocks]` were verified vs. timed out vs. missing-expected-provider vs. fetch-errored.
This matters most on the abort/timeout path: when an outer `AbortSignal.timeout(...)` fires mid-walk, the in-flight `fetch` throws `AbortError` and we lose which CID was current and which CIDs had already been verified successfully.
Concrete example from the dealbot consumer (FilOzone/dealbot):
- Deals are uploaded with rootCID + ~12 childBlocks. Outer timeout 60s, delayMs 2500ms, so each CID gets ~25 attempts.
- 7 of 8 SPs verify cleanly at p50 ~2.4s. SP6 (Hangzhou CN) verifies at p50 ~31s with 39% slow_verification rate; ~22 deals/24h time out at the 60s budget.
- All timeouts surface as a single generic message. We currently cannot distinguish \"rootCID never indexed\" from \"rootCID indexed but childBlock N never indexed.\"
- Investigation: https://github.com/FilOzone/dealbot/issues/473#issuecomment-4336343553
## Why this is currently low priority
1. In observed dealbot data, all timeouts are wall-clock SLO breaches, not selective \"missing expected provider\" failures. Per-CID granularity would not have shifted root-cause attribution (path quality between filecoinpin.contact / Reston VA and SP6 / Hangzhou CN, AS4134 ChinaNet, ICMP packet loss 24% from a peer test vantage).
2. At dealbot's current SLO budget (60s with 2.5s polls and 12 CIDs), only the first 4-5 CIDs get any attempts before the outer abort fires anyway — so child-CID granularity is not measurable end-to-end without also rebudgeting.
3. PR https://github.com/FilOzone/dealbot/pull/490 + https://github.com/FilOzone/dealbot/pull/491 close the immediate visibility gap that prompted dealbot#473 by carrying the existing single failureReason via `Error.cause`.
## Proposal (when prioritized)
Change return type from `Promise` to something like:
\`\`\`ts
type IpniValidationOutcome = {
verified: Array<{ cid: CID; matchedAt: number; attempts: number }>;
failed: Array<{
cid: CID;
reason:
| { type: 'timeout'; attempts: number }
| { type: 'missingProviders'; missingServiceUrls: string[]; actualMultiaddrs: string[] }
| { type: 'fetch'; message: string }
| { type: 'parse'; message: string }
| { type: 'http'; status: number; statusText?: string };
}>;
ipniIndexerUrl: string;
};
\`\`\`
Existing single-error throw can be preserved as a thin wrapper for callers that prefer the boolean shape, or replaced with always returning the outcome and letting callers decide whether to throw.
## Acceptance criteria
- [ ] `waitForIpniProviderResults` returns or exposes per-CID verified/failed lists.
- [ ] Abort during `setTimeout(delayMs)` (currently signal-unaware at https://github.com/filecoin-project/filecoin-pin/blob/main/src/core/utils/validate-ipni-advertisement.ts#L387) is preserved or fixed in the same change. Optional related fix: make the inter-attempt sleep signal-aware so abort overshoot is bounded by the in-flight fetch rather than by `delayMs`.
- [ ] Existing consumers that only need pass/fail can keep that behavior via a wrapper or a boolean accessor on the new return type.
- [ ] Tests cover: timeout-mid-walk, partial verification (some CIDs verified, some failed), all-CIDs-missing, single-CID success.
## Consumer signal
Open this when:
- A real dealbot incident appears where timeout reason \"some childBlock never indexed\" needs to be distinguished from \"rootCID never indexed.\"
- Or filecoin-pin gains another consumer that needs per-CID outcomes for its own product semantics.
Contributor guide
Assessment
This issue has not been assessed yet.