python / python/cpython

When accepting a socket connection and ERROR_NETNAME_DELETED occurs, leads this into a closing of the serving socket (BaseProactorEventLoop)

Aperta
#93,821 20 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

stdlib topic-asyncio type-bug
Lingua principale
Python
Stelle
77.2k
Fork
36k
Merge medio
1g 9h
PR unite (30g)
558

Descrizione

Hey Guys
Application Description

We discover on our uvicorn server sometimes that the listening socket is closing.
The Uvicorn get image uploads from cameras and sometimes or often releated to mobile network conditions the connection is lost.
And when a connection is lost on remote site, this error occurs sometimes.

I am not very deep into such low level python code so I tried it hard to find the root couse and this are my results.
I hope this results show's that there is sometings and we can talk about that or you can give me hints or help do resolve this issue.

thank you in advance

Bug/Error that occurs

The Error occurs on ov.getresult() (asyncio.windows_events.py:560)
OSError: [WinError 64] The specified network name is no longer available.

After that the server is unresponsive and the listening socket is closed. In that case we need to restart the service.

To me it looks like this happens when ov.getresult() is called and the remote host is disconnected.
When I just wrap this function in a try\except block the listening socket is not closing and uvicorn detects a connection lost.

Please let me know why just wrapping this into a try block resolve this issue,
because I don't know ;-)

Original

def finish_accept(trans, key, ov):
     ov.getresult() # Here happen the OSError
     # Use SO_UPDATE_ACCEPT_CONTEXT so getsockname() etc work.
     buf = struct.pack('@P', listener.fileno())
     conn.setsockopt(socket.SOL_SOCKET,
         _overlapped.SO_UPDATE_ACCEPT_CONTEXT, buf)
     conn.settimeout(listener.gettimeout())
     return conn, conn.getpeername()

Mod (No socket closings)

def finish_accept(trans, key, ov):
     try:
         ov.getresult()
     except OSError as Error:
         print("This helps", Error)
     # Use SO_UPDATE_ACCEPT_CONTEXT so getsockname() etc work.
     buf = struct.pack('@P', listener.fileno())
     conn.setsockopt(socket.SOL_SOCKET,
         _overlapped.SO_UPDATE_ACCEPT_CONTEXT, buf)
     conn.settimeout(listener.gettimeout())
     return conn, conn.getpeername()

Error without the try\catch block

Task exception was never retrieved
future: <Task finished name='Task-248' coro=<IocpProactor.accept.<locals>.accept_coro() done, defined at C:\Program Files\Python310\lib\asyncio\windows_events.py:570> exception=OSError(22, 'The specified network name is no longer available', None, 64, None) created at C:\Program Files\Python310\lib\asyncio\tasks.py:636>
source_traceback: Object created at (most recent call last):
  File "Y:\code_projects\pol\pol\server\test\brotle.py", line 31, in <module>
    asyncio.run(serve(Starlette(debug=True, routes=routes), config), debug=True)
  File "C:\Program Files\Python310\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "C:\Program Files\Python310\lib\asyncio\base_events.py", line 633, in run_until_complete
    self.run_forever()
  File "C:\Program Files\Python310\lib\asyncio\windows_events.py", line 321, in run_forever
    super().run_forever()
  File "C:\Program Files\Python310\lib\asyncio\base_events.py", line 600, in run_forever
    self._run_once()
  File "C:\Program Files\Python310\lib\asyncio\base_events.py", line 1888, in _run_once
    handle._run()
  File "C:\Program Files\Python310\lib\asyncio\events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "C:\Program Files\Python310\lib\asyncio\proactor_events.py", line 838, in loop
    f = self._proactor.accept(sock)
  File "C:\Program Files\Python310\lib\asyncio\windows_events.py", line 580, in accept
    tasks.ensure_future(coro, loop=self._loop)
  File "C:\Program Files\Python310\lib\asyncio\tasks.py", line 615, in ensure_future
    return _ensure_future(coro_or_future, loop=loop)
  File "C:\Program Files\Python310\lib\asyncio\tasks.py", line 636, in _ensure_future
    return loop.create_task(coro_or_future)
Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\asyncio\windows_events.py", line 573, in accept_coro
    await future
  File "C:\Program Files\Python310\lib\asyncio\windows_events.py", line 819, in _poll
    value = callback(transferred, key, ov)
  File "C:\Program Files\Python310\lib\asyncio\windows_events.py", line 561, in finish_accept
    ov.getresult()
OSError: [WinError 64] The specified network name is no longer available

Minimal Example to reproduce

  1. Start Server
  2. Start Client

Server (Minimal Reproducible Example)

from starlette.applications import Starlette
from starlette.responses import JSONResponse
from starlette.routing import Route

async def test_case(request):
    return JSONResponse({'hello': 'world'})

routes = [
    Route("/", endpoint=test_case, methods=['POST'])
]

##### Uvicorn Test
import uvicorn
uvicorn.run(
    Starlette(debug=True, routes=routes),
    reload=False,
    log_level="trace",
    host="0.0.0.0",
    port=8088,
)


##### Hypercorn Test
#import asyncio
#from hypercorn.config import Config
#from hypercorn.asyncio import serve

#config = Config()
#config.bind = ["localhost:8088"]
#asyncio.run(serve(Starlette(debug=True, routes=routes), config), debug=True)

Client (Minimal Reproducible Example)

import threading
import requests
import time
import sys


def test():
    for i in range(100):
        requests.post("http://127.0.0.1:8088/")

if __name__ == "__main__":
    for i in range(100):
        thread = threading.Thread(target=test, args=(), daemon=True)
        thread.start()
    time.sleep(1)
    sys.exit()

Tested with:
Uvicorn [x]
Hypercorn [x]

Environment
Python 3.10.5 (tags/v3.10.5:f377153, Jun 6 2022, 16:14:13) [MSC v.1929 64 bit (AMD64)] on win32

Linked PRs
  • gh-124032
  • gh-124779

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Direzione di ricerca

Inizia da asyncio/windows_events.py, in particolare da finish_accept e dalla chiamata ov.getresult() mostrata nel traceback. Riproduci il caso Windows Python 3.10.5 con il server e il client Starlette/Uvicorn forniti, quindi confronta il comportamento con le PR collegati gh-124032 e gh-124779; il lavoro è completato quando l’accept path gestisce la disconnect segnalata senza chiudere il listening socket.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
networking, operating-systems
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.