HelloZeroNet / HelloZeroNet/ZeroNet

Slow start due to quadratic time algorithms

Open
#2,770 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
JavaScript
Stars
18.8k
Forks
2.3k
PR merge metrics
No merged PRs in 30d

Description

### Step 1: Please describe your environment

* ZeroNet version: v0.7.2 (rev4555)
* Operating system: N/A
* Web browser: N/A
* Tor status: N/A
* Opened port: N/A
* Special configuration: N/A

### Step 2: Describe the problem:

ZeroNet takes about an hour to start with a few thousand sites downloaded.

The logs say:

```
[13:03:17] SiteManager Added new site: [address]
[13:03:17] Site:1A28C5..E1dw ContentDb init: 0.000s, found files: 0, sites: 8963
[13:03:17] Site:1A28C5..E1dw ContentDb not initialized, load files from filesystem...
[13:03:17] Site:1A28C5..E1dw Content.json not exist: ./data/[address]/content.json
[13:03:17] RateLimit Added to queue (2.75s left): Save sites.json
[13:03:17] SiteManager Updated merger sites in 0.601s
```
(repeat a few thousand times)

The 'Updated merger sites' seems to be the culprit. It's from `SiteManagerPlugin.updateMergerSites`, which scans all the downloaded sites, and is invoked every time a site is initialized. That takes quadratic time.

#### Steps to reproduce:

1. Create/download lots of sites
2. ???
3. PROFIT

#### Observed Results:

Quadratic time intiailization.

#### Expected Results:

Linear (ideally) time initialization.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at SiteManagerPlugin.updateMergerSites and trace where it is invoked during site initialization. Reproduce startup with a large collection of downloaded sites while reviewing the repeated “Updated merger sites” timings. Done means initialization no longer rescans all sites quadratically and startup time scales linearly or substantially better.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.