IPNS Pubsub Reprovider Duration (GC for unused topics)
- Dominant language
- Go
- Stars
- 17.1k
- Forks
- 3.2k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 11
Description
### TLDR
> Part of https://github.com/ipfs/go-ipfs/issues/6621, high impact: unbocks enabling fast IPNS, pubsub, and mitigating IPNS link rot by reproviding records
### Rationale
We want go-ipfs users in desktop use case (IPFS Desktop, Brave) and HTTP Gateways to have Pubsub and IPNS over pubsub enabled by default (#6621).
One of the blockers for enabling it by default for all our users is IPNS over pubsub listening to IPNS topic forever. When running on a server hosting HTTP Gateway (or a very active desktop client) resolving `/ipns/{libp2p-key}` will add topic listener that never expires.
### Proposed fix
Introduce usage-based GC for unused pubsub topics created by IPNS over pubsub.
We don't want the expiration to be too aggresive: there is a value in peers running "IPNS record reprovider" over the libp2p's fetch protocol. I think ~48h would match current expiration window of IPNS record in DHT + solve the problem of "IPNS rot" caused by someone's laptop going offline.
To implement the GC:
* go-ipfs should track when was the last time content path starting with `/ipns/{libp2p-key}` was resolved by namesys (`ipfs name resolve` or `ipfs resolve`)
* and unsubscribe from it after N hours since last lookup.
* make it configurable, details TBD:
* there should be an optional config flag with an implicit default (`OptionalDuration`), so we can adjust the default without impacting user's configs.
* we could just name it `Ipns.ReproviderDuration` if there is no point in making this ipns-specific – I believe we want to enable reproviding on DHT as well, not sure if there is any value in having separate `Ipns.DHTReproviderDuration` and `Ipns.PubsubReproviderDuration`
---------------------------------------------------------------
@schomatis' notes
# IPNS Pubsub Reprovider Duration (GC for unused topics)
https://github.com/ipfs/go-ipfs/issues/8586
Related spec: https://github.com/ipfs/specs/blob/master/naming/pubsub.md.
Most of the logic of interest (subscription handling) is in the `PubsubValueStore` structure in https://github.com/libp2p/go-libp2p-pubsub-router.
Also in the `IpnsResolver` in `github.com/ipfs/go-namesys@v0.4.0/routing.go`.
## Proposed solution
My biggest doubt is if we leave the subscription standing because we want to avoid unsubscribing and resubscribing repeatedly or is this just a technical debt.
If we actually want to leave the subscription standing then Lidel's proposal seems fine to me and I can add an independent goroutine in `NewPubsubValueStore` (similar to `go psValueStore.rebroadcast(ctx)` [here](https://github.com/libp2p/go-libp2p-pubsub-router/blob/1d5fc5c4ed11ec50a2ce08712f1fd6eb7e108130/pubsub.go#L108)) that periodically checks the last read/write (`PutValue`/`GetValue`) of each topic and unsubscribes after certain time has elapsed (and no one is watching).
If, on the other hand, the standing subscriptions are a technical debt then we should have a finer-grained control where we unsubscribe after each topic access (again, if no one is watching it).
## Simplest path to trigger a topic subscription for IPNS using the `ipfs name` commands
```
ipfs config --json Pubsub.Enabled true
ipfs config --json Ipns.UsePubsub true # Not working.
ipfs daemon --enable-namesys-pubsub --enable-pubsub-experiment &
# Do we need both?
ipfs name pubsub state
# enabled
# WARN: We will use the IPNS resolver ONLY for peer IDs arguments, not for any
# normal domain that will go through the DNS resolver (we have multiple
# resolvers through the `namesys.mpns` abstraction layer).
ipfs name resolve --nocache QmWdQmLNaG5rC3kLxJNjYU3E9a6tZ1nhM8kinAZenY8q3c
# (Random peer ID extracted from `ipfs swarm peers`)
# This will normally just hang (even without the `--stream` flag) trying to
# resolve. In the meanwhile we will have generated a subscription that can
# be examined through the `ipfs name pubsub` command. (We can even quickly
# cancel the command and the subscription will still remain there.)
ipfs name pubsub subs --ipns-base v0
# /ipns/QmWdQmLNaG5rC3kLxJNjYU3E9a6tZ1nhM8kinAZenY8q3c
# Same peer ID with the `/ipns/` prefix. The encoding argument
# `--ipns-base` is very important to not get some weird `base36`
# encoding that will obscure this relation.
# To cancel (remove the topic from the PubSub vault):
ipfs name pubsub cancel /ipns/QmWdQmLNaG5rC3kLxJNjYU3E9a6tZ1nhM8kinAZenY8q3c
# If the command is running (even without the `--stream` option) the operation
# will fail with `Error: key has active subscriptions` as the
# `PubsubValueStore.watching` still has the resolve command registered.
# FIXME: Correct description above: we always go through the `SearchValue` API
# which will first look for a local copy and if none exist "watch" for the IPNS
# topic.
# Not sure how the --stream and --nocache options of the `resolve` command map
# to this (but not important to look into).
```
Contributor guide
Assessment
This issue has not been assessed yet.