race condition when downloading same blob from multiple peers
Open
Nobody has claimed this yet.
area: blob exchange
type: bug
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 493
- PR merge metrics
- No merged PRs in 30d
Description
raise OSError(f"attempted to download blob twice from {peer_address}:{peer_port}") OSError: attempted to download blob twice from None:None
2019-07-08 11:29:05,276 INFO lbry.blob_exchange.client:155: downloaded 6118e6c6 from 73.162.215.156:3333 --- Logging error --- Traceback (most recent call last): File "c:\users\thoma\documents\lbry\lbry\lbry\blob_exchange\client.py", line 185, in download_blob self.blob, self.writer = blob, blob.get_blob_writer(self.peer_address, self.peer_port) File "c:\users\thoma\documents\lbry\lbry\lbry\blob\blob_file.py", line 205, in get_blob_writer raise OSError(f"attempted to download blob twice from {peer_address}:{peer_port}") OSError: attempted to download blob twice from None:None During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\logging\handlers.py", line 68, in emit if self.shouldRollover(record): File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\logging\handlers.py", line 184, in shouldRollover msg = "%s\n" % self.format(record) File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\logging\__init__.py", line 880, in format return fmt.format(record) File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\logging\__init__.py", line 619, in format record.message = record.getMessage() File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\logging\__init__.py", line 380, in getMessage msg = msg % self.args TypeError: %i format: a number is required, not NoneType Call stack: File "C:\Users\thoma\Documents\lbry\lbry-venv\Scripts\lbrynet-script.py", line 11, in <module> load_entry_point('lbry', 'console_scripts', 'lbrynet')() File "c:\users\thoma\documents\lbry\lbry\lbry\extras\cli.py", line 289, in main loop.run_until_complete(daemon.stop_event.wait()) File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 571, in run_until_complete self.run_forever() File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 539, in run_forever self._run_once() File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 1775, in _run_once handle._run() File "C:\Users\thoma\AppData\Local\Programs\Python\Python37-32\lib\asyncio\events.py", line 88, in _run self._context.run(self._callback, *self._args) File "c:\users\thoma\documents\lbry\lbry\lbry\blob_exchange\client.py", line 248, in request_blob return await protocol.download_blob(blob) File "c:\users\thoma\documents\lbry\lbry\lbry\blob_exchange\client.py", line 190, in download_blob log.warning("race happened downloading %s from %s:%i", blob_hash, self.peer_address, self.peer_port) Message: 'race happened downloading %s from %s:%i' Arguments: ('9e91e4e06d02c000b7202fb746422a42a86ba22b8fa5876d68f429c70029f70947177b9fa6f3754cb502a67f1da2d9b5', None, None) --- Logging error --- Traceback (most recent call last): File "c:\users\thoma\documents\lbry\lbry\lbry\blob_exchange\client.py", line 185, in download_blob self.blob, self.writer = blob, blob.get_blob_writer(self.peer_address, self.peer_port) File "c:\users\thoma\documents\lbry\lbry\lbry\blob\blob_file.py", line 205, in get_blob_writer raise OSError(f"attempted to download blob twice from {peer_address}:{peer_port}") OSError: attempted to download blob twice from None:None
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with lbry/blob_exchange/client.py at download_blob and lbry/blob/blob_file.py at get_blob_writer, then trace how concurrent peer requests reach those points. Reproduce the duplicate-download case and verify that concurrent requests no longer produce the reported race error or the None-valued logging failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100