filecoin-project / filecoin-project/filecoin-pin

waitForIpniProviderResults: surface per-CID outcomes for diagnostic visibility

Open
#417 1 comment 0 reactions 1 assignee Claimed by @SgtPooki View on GitHub
enhancement ready-for-work team/filecoin-pin team/fs-wg
Dominant language
TypeScript
Stars
26
Forks
21
Avg merge
5d 18h
Merged PRs (30d)
27

Description

## Problem

`waitForIpniProviderResults` returns `Promise` and on failure throws a single `Error` whose message references one CID (the last one checked). Consumers cannot tell, after the fact, which CIDs in `[rootCid, ...childBlocks]` were verified vs. timed out vs. missing-expected-provider vs. fetch-errored.

This matters most on the abort/timeout path: when an outer `AbortSignal.timeout(...)` fires mid-walk, the in-flight `fetch` throws `AbortError` and we lose which CID was current and which CIDs had already been verified successfully.

Concrete example from the dealbot consumer (FilOzone/dealbot):

- Deals are uploaded with rootCID + ~12 childBlocks. Outer timeout 60s, delayMs 2500ms, so each CID gets ~25 attempts.
- 7 of 8 SPs verify cleanly at p50 ~2.4s. SP6 (Hangzhou CN) verifies at p50 ~31s with 39% slow_verification rate; ~22 deals/24h time out at the 60s budget.
- All timeouts surface as a single generic message. We currently cannot distinguish \"rootCID never indexed\" from \"rootCID indexed but childBlock N never indexed.\"
- Investigation: https://github.com/FilOzone/dealbot/issues/473#issuecomment-4336343553

## Why this is currently low priority

1. In observed dealbot data, all timeouts are wall-clock SLO breaches, not selective \"missing expected provider\" failures. Per-CID granularity would not have shifted root-cause attribution (path quality between filecoinpin.contact / Reston VA and SP6 / Hangzhou CN, AS4134 ChinaNet, ICMP packet loss 24% from a peer test vantage).
2. At dealbot's current SLO budget (60s with 2.5s polls and 12 CIDs), only the first 4-5 CIDs get any attempts before the outer abort fires anyway — so child-CID granularity is not measurable end-to-end without also rebudgeting.
3. PR https://github.com/FilOzone/dealbot/pull/490 + https://github.com/FilOzone/dealbot/pull/491 close the immediate visibility gap that prompted dealbot#473 by carrying the existing single failureReason via `Error.cause`.

## Proposal (when prioritized)

Change return type from `Promise` to something like:

\`\`\`ts
type IpniValidationOutcome = {
verified: Array<{ cid: CID; matchedAt: number; attempts: number }>;
failed: Array<{
cid: CID;
reason:
| { type: 'timeout'; attempts: number }
| { type: 'missingProviders'; missingServiceUrls: string[]; actualMultiaddrs: string[] }
| { type: 'fetch'; message: string }
| { type: 'parse'; message: string }
| { type: 'http'; status: number; statusText?: string };
}>;
ipniIndexerUrl: string;
};
\`\`\`

Existing single-error throw can be preserved as a thin wrapper for callers that prefer the boolean shape, or replaced with always returning the outcome and letting callers decide whether to throw.

## Acceptance criteria

- [ ] `waitForIpniProviderResults` returns or exposes per-CID verified/failed lists.
- [ ] Abort during `setTimeout(delayMs)` (currently signal-unaware at https://github.com/filecoin-project/filecoin-pin/blob/main/src/core/utils/validate-ipni-advertisement.ts#L387) is preserved or fixed in the same change. Optional related fix: make the inter-attempt sleep signal-aware so abort overshoot is bounded by the in-flight fetch rather than by `delayMs`.
- [ ] Existing consumers that only need pass/fail can keep that behavior via a wrapper or a boolean accessor on the new return type.
- [ ] Tests cover: timeout-mid-walk, partial verification (some CIDs verified, some failed), all-CIDs-missing, single-CID success.

## Consumer signal

Open this when:
- A real dealbot incident appears where timeout reason \"some childBlock never indexed\" needs to be distinguished from \"rootCID never indexed.\"
- Or filecoin-pin gains another consumer that needs per-CID outcomes for its own product semantics.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.