Using `TaskPool::scope` inside a task can cause stack overflow.
- Dominant language
- Rust
- Stars
- 48.2k
- Forks
- 4.8k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 171
Description
## Version
main: 0410482c
## Problem
When calling `TaskPool::scope`, it allows you to spawn futures tied to a lifetime. To do this safely, the task must continue running on the thread until the futures are complete. This means at the very least, it must block. The current implementation does not block - instead it ticks the executor where the futures are spawned (in practice, the global executor). This means that rather than the thread idling, it goes and performs other work. This is in theory quite nice! We don't have a worker thread sitting around doing nothing, and we still "block" to maintain safety.
However, this results in a problem: what if the next work item that the thread picks up also does a `scope`? Well then it will add that to the stack, and then it will start ticking the executor again (because that task also needs to block), and then again the thread may pick up other work that starts a `scope`, and so on - in the thread's quest to try to stay busy, it took on too many `scope`s and causes a stack overflow.
## Additional info
This causes #15271. #15271 can be solved by just not using a TaskPool::scope (making the loading single-threaded), whereas this issue is the more general problem of any sort of task.
Contributor guide
Research direction
Start at TaskPool::scope and trace how it ticks the executor while a scoped task is blocked, then reproduce the nested-scope scenario described here and in #15271. Determine how nested scopes can preserve the required lifetimes without growing the call stack indefinitely; done means the workload no longer causes a stack overflow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100