aio-libs / aio-libs/aiopg

FileNotFoundError when connecting to postgres if fd is closed and then reopened

Aberta
#837 6 comentários 8 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
1.4k
Forks
170
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

I recently upgrade from python 3.6 to 3.8, and encountered a strange bug:

```python
File ".../venv/lib/python3.8/site-packages/aiopg/connection.py", line 151, in _ready
self._loop.add_writer(self._fileno, self._ready, weak_self)
File "/usr/lib/python3.8/asyncio/selector_events.py", line 337, in add_writer
return self._add_writer(fd, callback, *args)
File "/usr/lib/python3.8/asyncio/selector_events.py", line 296, in _add_writer
self._selector.modify(fd, mask | selectors.EVENT_WRITE,
File "/usr/lib/python3.8/selectors.py", line 389, in modify
self._selector.modify(key.fd, selector_events)
FileNotFoundError: [Errno 2] No such file or directory
```

The crash occurred when connecting to a database with `await aiopg.sa.create_engine(...)`. I was only able to reproduce it under certain circumstances: for example if I disabled SSL it would not happen, and if I used a .pgpass file rather than passing a password to `create_engine()` it would not happen.

Here's what the strace looked like at the time of the crash:
```
socket(AF_INET, SOCK_STREAM, IPPROTO_IP) = 10
setsockopt(10, SOL_TCP, TCP_NODELAY, [1], 4) = 0
fcntl(10, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(10, F_SETFL, O_RDWR|O_NONBLOCK) = 0
fcntl(10, F_SETFD, FD_CLOEXEC) = 0
setsockopt(10, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
connect(10, {sa_family=AF_INET, sin_port=htons(5432), sin_addr=inet_addr("X.X.X.X")}, 16) = -1 EINPROGRESS (Operation now in progress)
epoll_ctl(6, EPOLL_CTL_MOD, 10, {EPOLLIN|EPOLLOUT, {u32=10, u64=10}}) = -1 ENOENT (No such file or directory)
```

I discovered that what was happening was that under certain conditions, libpq will close and then reopen the socket, such that the fd underlying the aiopg connection is a new socket but has the same fd number. Turns out that in the [documentation](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-PQCONNECTSTARTPARAMS) they have a disclaimer that they are allowed to do this:

> Use PQsocket(conn) to obtain the descriptor of the socket underlying the database connection. (Caution: do not assume that the socket remains the same across PQconnectPoll calls.)

In python 3.6, the implementation of `_PollLikeSelector.modify` was to call `unregister()` and then `register()`. In python 3.7 they added a [patch](https://github.com/python/cpython/pull/1030) which changed the implementation: now it uses `epoll.modify()`. Whereas before, if we had replaced the socket with a new one with the same fd number, the unregister/register would still work, but now that it's a different socket, the modify causes us to do an `EPOLL_CTL_MOD` before `EPOLL_CTL_ADD`, returning `ENOENT`.

The bottom line is that libpq thinks that it's ok to replace the socket silently, and python doesn't. It seems that the best place to resolve this contradiction might be in aiopg. A possible workaround might be to detect that the socket has been replaced, and to remove the fd from the event loop and re-add it.

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.