theupdateframework / theupdateframework/python-tuf

Investigate/test fetcher retries

Open
#2,842 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.7k
Forks
304
Avg merge
1d 2h
Merged PRs (30d)
17

Description

I'm expecting urllib3 to retry 3 times on common http failures when using urllib3fetcher

There's an interesting failure in sigstore-probers where this does not seem to happen:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 754, in _error_catcher
    yield
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 879, in _raw_read
    data = self._fp_read(amt, read1=read1) if not fp_closed else b""
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 862, in _fp_read
    return self._fp.read(amt) if amt is not None else self._fp.read()
           ~~~~~~~~~~~~~^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/http/client.py", line 479, in read
    s = self.fp.read(amt)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/socket.py", line 719, in readinto
    return self._sock.recv_into(b)
           ~~~~~~~~~~~~~~~~~~~~^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/ssl.py", line 1304, in recv_into
    return self.read(nbytes, buffer)
           ~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/ssl.py", line 1138, in read
    return self._sslobj.read(len, buffer)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
TimeoutError: The read operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.13.4/x64/bin/tuf-on-ci-test-client", line 8, in <module>
    sys.exit(client())
             ~~~~~~^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf_on_ci/client.py", line 84, in client
    updater.refresh()
    ~~~~~~~~~~~~~~~^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf/ngclient/updater.py", line 170, in refresh
    self._load_timestamp()
    ~~~~~~~~~~~~~~~~~~~~^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf/ngclient/updater.py", line 425, in _load_timestamp
    data = self._download_metadata(
        Timestamp.type, self.config.timestamp_max_length
    )
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf/ngclient/updater.py", line 313, in _download_metadata
    return self._fetcher.download_bytes(url, length)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf/ngclient/fetcher.py", line 139, in download_bytes
    with self.download_file(url, max_length) as dl_file:
         ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/contextlib.py", line 141, in __enter__
    return next(self.gen)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf/ngclient/fetcher.py", line 102, in download_file
    for chunk in chunks:
                 ^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf/ngclient/urllib3_fetcher.py", line 105, in _chunks
    yield from response.stream(self.chunk_size)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 1066, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 955, in read
    data = self._raw_read(amt)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 878, in _raw_read
    with self._error_catcher():
         ~~~~~~~~~~~~~~~~~~~^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 759, in _error_catcher
    raise ReadTimeoutError(self._pool, None, "Read timed out.") from e  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='tuf-repo-cdn.sigstore.dev', port=443): Read timed out.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in tuf/ngclient/urllib3_fetcher.py, especially _chunks, and trace how tuf/ngclient/fetcher.py's download_file consumes the response stream. Reproduce the sigstore-probers timeout if possible and inspect urllib3's retry behavior for streaming reads. Done means the expected three retries on common HTTP failures are verified or the missing behavior is clearly identified with focused test coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.