Support multi-thread execution on multistage engine
Open
multi-stage
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
as a continuation of the discussion in #9615 that make the decision to not include multi-thread execution as part of the previous design review.
Mainly discuss the design for how to multi-thread execute in multi-stage engine, 2 initial thoughts
- execute operators in pipeline fashion with block boundary.
- execute partitioned blocks in parallel within each stage execution.
Discussion in previous threads:
https://github.com/apache/pinot/issues/9615#issuecomment-1282532520
Contributor guide
Assessment
This issue has not been assessed yet.