`num_py_executors` thread pool visibility
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
`ir_context.to_thread(...)` (`dsl/ir.py`) dispatches blocking/GPU-launching work onto a `ThreadPoolExecutor` sized by `num_py_executors` (default 8, constructed the same way in `engine/spmd.py`, `engine/dask.py`, and `engine/ray.py`). It's not I/O-specific: `read_chunk` uses it, but so does join-partition compute (`actor_graph/join.py`), window/`Over` evaluation (`actor_graph/over.py`), generic per-chunk `do_evaluate` for other IR node types (`actor_graph/nodes.py`), shared per-chunk evaluation/concat helpers used by groupby and sort (`actor_graph/utils.py`), and AllGather result concatenation (`actor_graph/collectives/allgather.py`). It's one process-wide pool shared by every actor type in the query, not just scan nodes. There's no visibility into whether it's saturated or idle today.
This is a distinct layer from both rapidsai/kvikio#1044, which executes byte-range fetches inside a `read_chunk` call specifically, and rapidsai/rapidsmpf#1169, which resumes the coroutine that *calls* `to_thread` in the first place. An I/O task can be slow to start for any of three separate reasons stacked on top of each other:
- its coroutine isn't being resumed
- its coroutine is resumed but has to wait for a free `num_py_executors` thread (now contended by every other actor type too, not just other scans)
- it gets a thread immediately but the read itself is slow
Right now none of the three are distinguishable. This sub-issue closes the second one. The first is closed by rapidsai/rapidsmpf#1169, and the third by #23811 and #23812, which already split `read_chunk`'s own timing into admission-wait vs. execution time once a thread is actually running it.
Add the same kind of queue/running count Python's `ThreadPoolExecutor` doesn't expose natively (it only exposes `_work_queue.qsize()` as a private implementation detail, worth wrapping rather than relying on that directly).
Contributor guide
Research direction
Start in dsl/ir.py at ir_context.to_thread, then compare ThreadPoolExecutor construction in engine/spmd.py, engine/dask.py, and engine/ray.py. Review the named actor_graph call sites to understand the shared pool's usage. Done means exposing queue and running counts so thread-wait contention can be distinguished from coroutine scheduling and read execution time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100