ipfs / ipfs/kubo

Unified Provide Interface for Content Routers

Open
#10,097 7 comments 0 reactions 0 assignees View on GitHub
effort/weeks exp/expert kind/feature kind/maintenance P2
Dominant language
Go
Stars
17.1k
Forks
3.2k
Avg merge
3d 18h
Merged PRs (30d)
11

Description

### Checklist

- [X] My issue is specific & actionable.
- [X] I am not suggesting a protocol enhancement.
- [X] I have searched on the [issue tracker](https://github.com/ipfs/kubo/issues?q=is%3Aissue) for my issue.

### Description

## Problem Statement:

Currently, Kubo is responsible for managing the DHT's provide and reprovide operations. However, with the evolution of Content Routers beyond just the DHT, it's evident that the existing mechanism is not optimal. The reasons are:

1. The reprovide strategy Kubo uses was mainly designed for DHTs and is not always suitable for the newer Content Routers such as [IPNI](https://docs.ipfs.tech/concepts/ipni/) which is using a different advertising mechanism.
2. The DHT cannot optimize its reprovide strategy as it doesn't have a direct insight into the content that needs to be republished.

## Proposed Solution

To better streamline the providing mechanism across different content routers, we propose a unified interface that shifts the responsibility from Kubo to the individual content routers. The proposed interface includes:

* `StartProvide(CIDs)`: Instructs the content router to begin advertising that the Kubo node is storing the specified CIDs. This advertisement (or republishing) should continue until a `StopProvide` is invoked for these CIDs.
* `StopProvide(CIDs)`: Commands the content router to cease the advertisement for the given CIDs.
* `ListProvides`: Returns the list of CIDs currently being advertised by the content router.

## Benefits

1. Flexibility: With a generic interface, different content routers can easily integrate with Kubo without being tied to a DHT-specific strategy.
2. Optimization Opportunities: Allows the DHT and other content routers to implement their own specific provide strategies, optimized for their use cases. In the DHT, this change of interface is necessary to implement [Reprovide Sweep](https://pl-strflt.notion.site/DHT-Reprovide-Sweep-3108adf04e9d4086bafb727b17ae033d) ([IPFS Thing 2023 presentation](https://www.youtube.com/watch?v=bXaL64fp55c)), allowing a resource efficient reprovide strategy, enabling large content providers to advertise content to the DHT.
3. Clarity of Responsibilities: Removing the responsibility from Kubo makes the system modular, allowing each component to focus on its core functionality.

## Feedback and Collaboration

The proposed interface is just a draft for now. The goal of this issue is to gather feedback and start a public discussion about specific interface needs for different content routers, especially IPNI and the DHT. This issue will probably be followed up by an IPIP in [ipfs/specs](https://github.com/ipfs/specs), once we have listed the requirements of all (known) Content Routers.

## References

* [IPFS Thing 2023 Unconf session notes](https://pl-strflt.notion.site/Content-Router-Interfaces-Notes-IPFS-Thing-81c829472a3b43a0b325f003de257409)

cc: @masih @ischasny @aschmahmann @Jorropo @dennis-tra @iand

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.