ipfs / ipfs/kubo

IPNS Pubsub Reprovider Duration (GC for unused topics)

Open
#8,586 9 comments 0 reactions 1 assignee Assigned to @schomatis View on GitHub
kind/enhancement topic/ipns
Dominant language
Go
Stars
17.1k
Forks
3.2k
Avg merge
3d 18h
Merged PRs (30d)
11

Description

### TLDR

> Part of https://github.com/ipfs/go-ipfs/issues/6621, high impact: unbocks enabling fast IPNS, pubsub, and mitigating IPNS link rot by reproviding records

### Rationale

We want go-ipfs users in desktop use case (IPFS Desktop, Brave) and HTTP Gateways to have Pubsub and IPNS over pubsub enabled by default (#6621).

One of the blockers for enabling it by default for all our users is IPNS over pubsub listening to IPNS topic forever. When running on a server hosting HTTP Gateway (or a very active desktop client) resolving `/ipns/{libp2p-key}` will add topic listener that never expires.

### Proposed fix

Introduce usage-based GC for unused pubsub topics created by IPNS over pubsub.

We don't want the expiration to be too aggresive: there is a value in peers running "IPNS record reprovider" over the libp2p's fetch protocol. I think ~48h would match current expiration window of IPNS record in DHT + solve the problem of "IPNS rot" caused by someone's laptop going offline.

To implement the GC:
* go-ipfs should track when was the last time content path starting with `/ipns/{libp2p-key}` was resolved by namesys (`ipfs name resolve` or `ipfs resolve`)
* and unsubscribe from it after N hours since last lookup.
* make it configurable, details TBD:
* there should be an optional config flag with an implicit default (`OptionalDuration`), so we can adjust the default without impacting user's configs.
* we could just name it `Ipns.ReproviderDuration` if there is no point in making this ipns-specific – I believe we want to enable reproviding on DHT as well, not sure if there is any value in having separate `Ipns.DHTReproviderDuration` and `Ipns.PubsubReproviderDuration`

---------------------------------------------------------------

@schomatis' notes

# IPNS Pubsub Reprovider Duration (GC for unused topics)

https://github.com/ipfs/go-ipfs/issues/8586

Related spec: https://github.com/ipfs/specs/blob/master/naming/pubsub.md.

Most of the logic of interest (subscription handling) is in the `PubsubValueStore` structure in https://github.com/libp2p/go-libp2p-pubsub-router.

Also in the `IpnsResolver` in `github.com/ipfs/go-namesys@v0.4.0/routing.go`.

## Proposed solution

My biggest doubt is if we leave the subscription standing because we want to avoid unsubscribing and resubscribing repeatedly or is this just a technical debt.

If we actually want to leave the subscription standing then Lidel's proposal seems fine to me and I can add an independent goroutine in `NewPubsubValueStore` (similar to `go psValueStore.rebroadcast(ctx)` [here](https://github.com/libp2p/go-libp2p-pubsub-router/blob/1d5fc5c4ed11ec50a2ce08712f1fd6eb7e108130/pubsub.go#L108)) that periodically checks the last read/write (`PutValue`/`GetValue`) of each topic and unsubscribes after certain time has elapsed (and no one is watching).

If, on the other hand, the standing subscriptions are a technical debt then we should have a finer-grained control where we unsubscribe after each topic access (again, if no one is watching it).

## Simplest path to trigger a topic subscription for IPNS using the `ipfs name` commands

```
ipfs config --json Pubsub.Enabled true
ipfs config --json Ipns.UsePubsub true # Not working.
ipfs daemon --enable-namesys-pubsub --enable-pubsub-experiment &
# Do we need both?

ipfs name pubsub state
# enabled

# WARN: We will use the IPNS resolver ONLY for peer IDs arguments, not for any
# normal domain that will go through the DNS resolver (we have multiple
# resolvers through the `namesys.mpns` abstraction layer).

ipfs name resolve --nocache QmWdQmLNaG5rC3kLxJNjYU3E9a6tZ1nhM8kinAZenY8q3c
# (Random peer ID extracted from `ipfs swarm peers`)
# This will normally just hang (even without the `--stream` flag) trying to
# resolve. In the meanwhile we will have generated a subscription that can
# be examined through the `ipfs name pubsub` command. (We can even quickly
# cancel the command and the subscription will still remain there.)

ipfs name pubsub subs --ipns-base v0
# /ipns/QmWdQmLNaG5rC3kLxJNjYU3E9a6tZ1nhM8kinAZenY8q3c
# Same peer ID with the `/ipns/` prefix. The encoding argument
# `--ipns-base` is very important to not get some weird `base36`
# encoding that will obscure this relation.

# To cancel (remove the topic from the PubSub vault):
ipfs name pubsub cancel /ipns/QmWdQmLNaG5rC3kLxJNjYU3E9a6tZ1nhM8kinAZenY8q3c
# If the command is running (even without the `--stream` option) the operation
# will fail with `Error: key has active subscriptions` as the
# `PubsubValueStore.watching` still has the resolve command registered.

# FIXME: Correct description above: we always go through the `SearchValue` API
# which will first look for a local copy and if none exist "watch" for the IPNS
# topic.
# Not sure how the --stream and --nocache options of the `resolve` command map
# to this (but not important to look into).
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.