libp2p / libp2p/universal-connectivity

Rust peer exhausting resources

Open
#135 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

rust
Dominant language
TypeScript
Stars
167
Forks
75
PR merge metrics
No merged PRs in 30d

Description

Problem

Every since https://github.com/libp2p/universal-connectivity/pull/114 was merged and the Rust peer is now part of the IPFS Kademlia DHT, it's resource consumption has spiked significantly. It's running on a pretty beefy machine (32GB RAM), yet manages to eat up the CPU cycles from the moment it starts:

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
   1687 root      20   0 1384924   1.0g  33812 S 650.3   3.3  11:30.25 rust-libp2p-web
   1718 root      20   0 2651788 131636  22300 S   2.7   0.4   0:11.17 go-peer

Especially when compared to the go-peer (which also joins the IPFS DHT as a DHT Server)

Potential solutions

It's probably sensible to figure out why the Rust peer is consuming so much resources in order to come up with solutions. We know that this problem started every since it joined the IPFS DHT and connected to the bootstrap nodes.

It may be sensible to instrument the code to figure out why this is happening, in addition to adding some connection limits and having a look at https://github.com/libp2p/rust-libp2p/tree/master/misc/server as a comparison rust-libp2p DHT Server.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the resource spike after the Rust peer joins the IPFS Kademlia DHT and connects to bootstrap nodes. Instrument the Rust peer to identify the source of CPU and memory use, and compare its setup with rust-libp2p's misc/server example. Done means the cause is identified and a concrete mitigation, such as connection limits, is validated against the reported behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, rust
Domain
networking, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.