Bundle gather calls in as_completed(..., with_results)
Open
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 778
- Avg merge
- 2h 50m
- Merged PRs (30d)
- 3
Description
Currently `as_completed(..., with_results=True)` can generate *many* gather calls, which can overwhelm the scheduler a bit. We might instead batch these gather calls if many come in before other gather calls return. This would also help to avoid creating very many file descriptors.
Alternativley this could also be handled at a lower level in gather itself.
Contributor guide
Assessment
This issue has not been assessed yet.