[BUG] cudf-polars thread-offload uses hardware-determined number of threads
Open
bug
cudf-polars
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Describe the bug**
In the cudf-polars executor we use `asyncio.to_thread` to multi-thread the coroutines. This uses the "default" executor from the current event loop. But we have no control over how many threads that uses.
We might imagine setting the threadpoolexecutor the user must provide as the thread pool for asyncio but that doesn't work because asyncio _shuts it down_ when the `asyncio.run(...)` call ends. Argh!
So I think we should implement our own `to_thread`-like thing.
Contributor guide
Assessment
This issue has not been assessed yet.