theupdateframework / theupdateframework/python-tuf

Investigate/test fetcher retries

Aperta
#2,842 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Lingua principale
Python
Stelle
1.7k
Fork
304
Merge medio
1g 2h
PR unite (30g)
17

Descrizione

I'm expecting urllib3 to retry 3 times on common http failures when using urllib3fetcher

There's an interesting failure in sigstore-probers where this does not seem to happen:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 754, in _error_catcher
    yield
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 879, in _raw_read
    data = self._fp_read(amt, read1=read1) if not fp_closed else b""
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 862, in _fp_read
    return self._fp.read(amt) if amt is not None else self._fp.read()
           ~~~~~~~~~~~~~^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/http/client.py", line 479, in read
    s = self.fp.read(amt)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/socket.py", line 719, in readinto
    return self._sock.recv_into(b)
           ~~~~~~~~~~~~~~~~~~~~^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/ssl.py", line 1304, in recv_into
    return self.read(nbytes, buffer)
           ~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/ssl.py", line 1138, in read
    return self._sslobj.read(len, buffer)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
TimeoutError: The read operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.13.4/x64/bin/tuf-on-ci-test-client", line 8, in <module>
    sys.exit(client())
             ~~~~~~^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf_on_ci/client.py", line 84, in client
    updater.refresh()
    ~~~~~~~~~~~~~~~^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf/ngclient/updater.py", line 170, in refresh
    self._load_timestamp()
    ~~~~~~~~~~~~~~~~~~~~^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf/ngclient/updater.py", line 425, in _load_timestamp
    data = self._download_metadata(
        Timestamp.type, self.config.timestamp_max_length
    )
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf/ngclient/updater.py", line 313, in _download_metadata
    return self._fetcher.download_bytes(url, length)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf/ngclient/fetcher.py", line 139, in download_bytes
    with self.download_file(url, max_length) as dl_file:
         ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/contextlib.py", line 141, in __enter__
    return next(self.gen)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf/ngclient/fetcher.py", line 102, in download_file
    for chunk in chunks:
                 ^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/tuf/ngclient/urllib3_fetcher.py", line 105, in _chunks
    yield from response.stream(self.chunk_size)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 1066, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 955, in read
    data = self._raw_read(amt)
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 878, in _raw_read
    with self._error_catcher():
         ~~~~~~~~~~~~~~~~~~~^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/contextlib.py", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "/opt/hostedtoolcache/Python/3.13.4/x64/lib/python3.13/site-packages/urllib3/response.py", line 759, in _error_catcher
    raise ReadTimeoutError(self._pool, None, "Read timed out.") from e  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='tuf-repo-cdn.sigstore.dev', port=443): Read timed out.

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Inizia da tuf/ngclient/urllib3_fetcher.py, in particolare da _chunks, e segui il modo in cui download_file di tuf/ngclient/fetcher.py consuma il flusso della risposta. Riproduci il timeout di sigstore-probers, se possibile, e analizza il comportamento di retry di urllib3 per le letture in streaming. Il lavoro è completo quando vengono verificati i tre retry previsti per gli errori HTTP comuni oppure il comportamento mancante viene identificato chiaramente con una copertura di test mirata.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
networking
Tipo di issue
Bug
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.