lbryio / lbryio/lbry-sdk

Concurrent download issues / potential timeouts

Open
#2,189 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area: downloads type: bug
Dominant language
Python
Stars
7.2k
Forks
493
PR merge metrics
No merged PRs in 30d

Description

The Issue

I was able to reproduce the issues @jackrobison was seeing with concurrent downloads. To me it seems that the same peer won't be available for the 2nd of the 2 concurrent downloads, and the 2nd download won't start until the first completes. I ran a script where 2 downloads would start concurrently and then compared it to a single download - the results were consistent.

The 2nd download would never use the same peers the first did (i.e. not connect to my PC / reflector). If I ran that same download on it's own, it would. On slower connections, this would most likely cause timeouts on the 2nd download.

The first and third runs are with 2 downloads, the 2nd/4th are with 1. This was done with lbry://bitcoin and lbry://one

Starting requests
get: 3141.890ms
get 2: 25644.097ms

C:\Users\thoma\Documents\timing-scripts>node script.js
Starting requests
get: 3176.332ms

C:\Users\thoma\Documents\timing-scripts>node script.js
Starting requests
get: 3655.100ms
get 2: 25320.714ms

C:\Users\thoma\Documents\timing-scripts>node script.js

Starting requests
resolve: 163.547ms
get: 3227.946ms

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the timing-scripts reproduction using node script.js and compare the recorded single-download and concurrent-download timings for lbry://bitcoin and lbry://one. Investigate why the second download cannot use peers available to the first; done means concurrent downloads can access eligible peers without the observed delay or timeout risk.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.