apache / apache/arrow

[C++] Generalize thread pool to allow for different queuing strategies / worker loops

Open
#28,608 1 comment 0 reactions 0 assignees View on GitHub
Component: C++ Type: task
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

Rather than duplicate everything in arrow::internal::ThreadPool when creating a work stealing thread pool we should make arrow::internal::ThreadPool a base class that can be extended by specific strategies.

This PR is simply to split the thread pool into a base class and a "SimpleThreadPool" implementation which uses a single queue of tasks and locks (i.e. what we have today).

**Reporter**: [Weston Pace](https://issues.apache.org/jira/browse/ARROW-12878) / @westonpace
#### Related issues:
- [[C++] Make ThreadPool task ordering configurable](https://github.com/apache/arrow/issues/24915) (relates to)
#### PRs and other links:
- [GitHub Pull Request #10401](https://github.com/apache/arrow/pull/10401)

**Note**: *This issue was originally created as [ARROW-12878](https://issues.apache.org/jira/browse/ARROW-12878). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start by reviewing arrow::internal::ThreadPool and the linked pull request #10401 to understand the proposed split. The work is done when ThreadPool serves as a base class and SimpleThreadPool preserves the current single-queue, lock-based behavior without duplicating the implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.