HelloZeroNet / HelloZeroNet/ZeroNet
Slow start due to quadratic time algorithms
- Dominant language
- JavaScript
- Stars
- 18.8k
- Forks
- 2.3k
- PR merge metrics
- No merged PRs in 30d
Description
### Step 1: Please describe your environment
* ZeroNet version: v0.7.2 (rev4555)
* Operating system: N/A
* Web browser: N/A
* Tor status: N/A
* Opened port: N/A
* Special configuration: N/A
### Step 2: Describe the problem:
ZeroNet takes about an hour to start with a few thousand sites downloaded.
The logs say:
```
[13:03:17] SiteManager Added new site: [address]
[13:03:17] Site:1A28C5..E1dw ContentDb init: 0.000s, found files: 0, sites: 8963
[13:03:17] Site:1A28C5..E1dw ContentDb not initialized, load files from filesystem...
[13:03:17] Site:1A28C5..E1dw Content.json not exist: ./data/[address]/content.json
[13:03:17] RateLimit Added to queue (2.75s left): Save sites.json
[13:03:17] SiteManager Updated merger sites in 0.601s
```
(repeat a few thousand times)
The 'Updated merger sites' seems to be the culprit. It's from `SiteManagerPlugin.updateMergerSites`, which scans all the downloaded sites, and is invoked every time a site is initialized. That takes quadratic time.
#### Steps to reproduce:
1. Create/download lots of sites
2. ???
3. PROFIT
#### Observed Results:
Quadratic time intiailization.
#### Expected Results:
Linear (ideally) time initialization.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at SiteManagerPlugin.updateMergerSites and trace where it is invoked during site initialization. Reproduce startup with a large collection of downloaded sites while reviewing the repeated “Updated merger sites” timings. Done means initialization no longer rescans all sites quadratically and startup time scales linearly or substantially better.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100