python / python/cpython

Unclosed socket in urllib when ftp request times out after connect

Aperta
#140,691 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

stdlib topic-IO type-bug
Lingua principale
Python
Stelle
77.2k
Fork
35.9k
Metriche PR in attesa

Descrizione

Bug report

Bug description:

Spotted in the log of "Log from "buildbot/ARM64 macOS 3.x"; this doesn't fix the ftp test but when the operation times out in the test CPython should close the socket

This seems to be originating here:
https://github.com/python/cpython/blob/f5394c257ce4118394fe5ff561a2b0ad238a8d65/Lib/urllib/request.py#L1539-L1556

In particular, fw = self.connect_ftp succeeds which means fw contains an open connection but fp, retrlen = fw.retrfile(file, type) fails and the except doesn't result in a fw.close().

Log from "buildbot/ARM64 macOS 3.x" 

Exception ignored while finalizing socket <socket.socket fd=6, family=2, type=1, proto=6, laddr=('192.168.0.101', 57993), raddr=('68.183.26.59', 21)>:
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1546, in ftp_open
    fp, retrlen = fw.retrfile(file, type)
                  ~~~~~~~~~~~^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1826, in retrfile
    conn, retrlen = self.ftp.ntransfercmd(cmd)
                    ~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/ftplib.py", line 354, in ntransfercmd
    conn = socket.create_connection((host, port), self.timeout,
                                    source_address=self.source_address)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 879, in create_connection
    raise exceptions[0]
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 864, in create_connection
    sock.connect(sa)
    ~~~~~~~~~~~~^^^^
TimeoutError: [Errno 60] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 20, in _retry_thrice
    return func(*args, **kwargs)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 187, in urlopen
    return opener.open(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1556, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: [Errno 60] Operation timed out>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 22, in _retry_thrice
    last_exc = e
    ^^^^^^^^
ResourceWarning: unclosed <socket.socket fd=6, family=2, type=1, proto=6, laddr=('192.168.0.101', 57993), raddr=('68.183.26.59', 21)>
Warning -- Unraisable exception
Exception ignored while finalizing socket <socket.socket fd=8, family=2, type=1, proto=6, laddr=('192.168.0.101', 58055), raddr=('68.183.26.59', 21)>:
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1546, in ftp_open
    fp, retrlen = fw.retrfile(file, type)
                  ~~~~~~~~~~~^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1826, in retrfile
    conn, retrlen = self.ftp.ntransfercmd(cmd)
                    ~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/ftplib.py", line 354, in ntransfercmd
    conn = socket.create_connection((host, port), self.timeout,
                                    source_address=self.source_address)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 879, in create_connection
    raise exceptions[0]
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/socket.py", line 864, in create_connection
    sock.connect(sa)
    ~~~~~~~~~~~~^^^^
TimeoutError: [Errno 60] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 20, in _retry_thrice
    return func(*args, **kwargs)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 187, in urlopen
    return opener.open(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/urllib/request.py", line 1556, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: [Errno 60] Operation timed out>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_urllib2net.py", line 22, in _retry_thrice
    last_exc = e
    ^^^^^^^^
ResourceWarning: unclosed <socket.socket fd=8, family=2, type=1, proto=6, laddr=('192.168.0.101', 58055), raddr=('68.183.26.59', 21)>

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs
  • gh-140694
  • gh-140835
  • gh-141555
  • gh-141657

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Leggi ftp_open in Lib/urllib/request.py, in particolare connect_ftp e il percorso di eccezione di retrfile descritto nel report. Controlla test/test_urllib2net.py e verifica che il caso di timeout non lasci più un socket non chiuso né emetta una ResourceWarning.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
networking
Tipo di issue
Bug
Difficoltà
2/5
Tempo stimato
1-3 ore
Stato di attività
Ferma
Chiarezza
Specificata chiaramente
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.