MagicStack / MagicStack/asyncpg

Concurrent queries on single connection

Open
#738 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8.1k
Forks
468
PR merge metrics
No merged PRs in 30d

Description

* **asyncpg version**: 0.22.0
* **PostgreSQL version**: 13.2
* **Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
the issue with a local PostgreSQL install?**: local PostgreSQL install
* **Python version**: 3.9
* **Platform**: centos
* **Do you use pgbouncer?**: No
* **Did you install asyncpg with pip?**: Yes
* **If you built asyncpg locally, which version of Cython did you use?**:
* **Can the issue be reproduced under both asyncio and
[uvloop](https://github.com/magicstack/uvloop)?**: uvloop

from asyncpg.exceptions import InterfaceError

class CursorIterator:
def __init__(self, connection, prefetch, portal_name):
if prefetch <= 0:
raise InterfaceError("prefetch argument must be greater than zero")

self._connection = connection
self._prefetch = prefetch
self._portal_name = portal_name
self.rows = []

def __aiter__(self):
return self

async def __anext__(self):
if not self.rows:
self.rows = await self._connection.fetch(
f"FETCH {self._prefetch} FROM {self._portal_name}"
)

if self.rows:
return self.rows.pop(0)

raise StopAsyncIteration


class Cursor:
def __init__(self, connection, query, *args, prefetch=None):
self._connection = connection
self._args = args
self._prefetch = prefetch
self._query = query
self._portal_name = connection._get_unique_id("portal")

async def __aenter__(self):
await self._connection.execute(
f"DECLARE {self._portal_name} NO SCROLL CURSOR WITH HOLD FOR {self._query}",
*self._args,
)

prefetch = 50 if self._prefetch is None else self._prefetch
return CursorIterator(self._connection, prefetch, self._portal_name)

async def __aexit__(self, *args):
await self._connection.execute(f"CLOSE {self._portal_name}")

Several queries are run concurrently and I receive the following error. What can I do to fix the problem ?

File "asyncpg/protocol/protocol.pyx", line 321, in query
File "asyncpg/protocol/protocol.pyx", line 684, in asyncpg.protocol.protocol.BaseProtocol._check_state
asyncpg.exceptions._base.InterfaceError: cannot perform operation: another operation is in progress

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading asyncpg/protocol/protocol.pyx around BaseProtocol._check_state and compare it with the supplied Cursor and concurrent-query example. Check whether the same connection is being used concurrently and determine the expected behavior or documentation scope; done should be a confirmed reproduction with an agreed maintainer direction.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.