graphql-python / graphql-python/graphql-core

Dataloader in multi-threaded environments

Offen
#71 10 Kommentare 0 Reaktionen 1 zugewiesene Person Beansprucht von @Cito Auf GitHub ansehen
discussion help wanted investigate
Vorherrschende Sprache
Python
Sterne
531
Forks
146
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

Hi,

I've been thinking a bit about how we could implement the Dataloader pattern in v3 while still running in multi-threaded mode. Since v3 does not support Syrus's [Promise library](https://github.com/syrusakbary/promise), we need to come up with a story for batching in async mode, as well as in multi-threaded environments. There are many libraries that do not support `asyncio` and there [are many cases](https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/) where it does not make sense to go fully async.

As far as I understand, the only way to batch resolver calls from a [single frame of execution](https://github.com/graphql/dataloader/blob/master/src/index.js#L220-L222) would be to use [`loop.call_soon`](https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.call_soon). But since `asyncio` is [not threadsafe](https://docs.python.org/3/library/asyncio-dev.html#concurrency-and-multithreading), that means we would need to run a separate event loop in each worker thread. We would need to wrap the `graphql` call with something like this:

```py
def run_batched_query(...):
loop = asyncio.new_event_loop()
execution_future = graphql(...)
loop.run_until_complete(result_future)
return execution_future.result()
```

Is that completely crazy? If yes, do you see a less hacky way? I'm not very familiar with `asyncio` so I would love to get feedback.

Cheers

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.