MagicStack / MagicStack/asyncpg
Infinite hang on leaving conn.transaction() block
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 468
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I am experiencing hangs at aexit of Transaction context manager, with traceback as follows:
...
File "/usr/local/lib/python3.7/dist-packages/asyncpg/transaction.py", line 89, in __aexit__
await self.__rollback()
File "/usr/local/lib/python3.7/dist-packages/asyncpg/transaction.py", line 198, in __rollback
await self._connection.execute(query)
File "/code/callmeback/detail/log_helpers/postgre_conn.py", line 53, in execute
return await super(LoggingConn, self).execute(query, *args, timeout=timeout)
File "/usr/local/lib/python3.7/dist-packages/asyncpg/connection.py", line 273, in execute
return await self._protocol.query(query, timeout)
Default command timeout is set for connection:
>>> conn = frame.f_locals['self']
>>> conn._config
ConnectionConfiguration(command_timeout=10.0, statement_cache_size=100, max_cached_statement_lifetime=300, max_cacheable_statement_size=15360)
>>> conn._protocol._get_timeout(None)
10.0
So I expect self._protocol.query() to throw TimeoutError of some kind, but related coroutine hanged for three days with no progress.
I will happily provide you with any additional information, if I can. My app has aiomonitor onboard, so if any info reachable that way will help you - please let me know.
I tried to fiddle with conn._protocol, but almost nothing of its state is reachable from aioconsole since it's Cython.
My suspection is that exception that leads to rollback is coming from in-transaction query that somehow leads connection in such a state that rollback() cannot be processed normally. I will provide you with information about exception once that problem triggers next time.
- asyncpg version: 0.18.3
- PostgreSQL version: 10.7
- Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
the issue with a local PostgreSQL install?: - Python version: 3.7
- Platform: Ubuntu 16.04
- Do you use pgbouncer?: Yes (session-level pooling)
- Did you install asyncpg with pip?: Yes
- If you built asyncpg locally, which version of Cython did you use?: --
- Can the issue be reproduced under both asyncio and
uvloop?: Yes, I am using uvloop
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with asyncpg/transaction.py at aexit and rollback, then follow connection.py execute() into _protocol.query(). Reproduce the hang with PostgreSQL 10.7, session-level pgbouncer pooling, and uvloop if possible; done means the rollback path either completes or surfaces the configured timeout instead of hanging.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100