ipfs / ipfs/kubo

Performance indexing "big directories"

Open
#7,588 3 comments 0 reactions 0 assignees View on GitHub
kind/bug need/triage
Dominant language
Go
Stars
17.1k
Forks
3.2k
Avg merge
3d 18h
Merged PRs (30d)
11

Description

#### Version information:

$ ipfs version --all
go-ipfs version: 0.6.0
Repo version: 10
System version: amd64/linux
Golang version: go1.14.4

#### Description:

Usage:
* Add a folder containing 35GiB of files across ~7,108 files.
* Documents only exist on a single node with a 200 Mbit up/down internet connection
* Navigate to folder on a gateway such as gateway.ipfs.io (success)
* Enter the folder (index) with 7,108 files. Timeout on gateway.

Does the IPFS gateway need to transfer a *large* amount of data to get a directory index?

The IPFS daemon has been running about 30Mbit/s for a while now.
![bandwidth](https://user-images.githubusercontent.com/100110/89788450-caeab700-dae4-11ea-8aa0-1db136eb0081.png)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.