facebookexperimental / facebookexperimental/libunifex
Allow for `bulk_` operations to utilize multiple threads
- Dominant language
- C++
- Stars
- 1.7k
- Forks
- 210
- PR merge metrics
- No merged PRs in 30d
Description
hello @ericniebler @kirkshoop ! Friendly pinging you on this just because i saw your names associated with recent activity :)
I am trying to come up with an example where I have a vector and would like to schedule async operations that operate
on parts of the data independently. For the time being, let's assume that all operations are "the same".
I would like to do this via a thread pool in parallel since I know that all operations are independent.
It seems to me that the goto choice would be to use `bulk_` operations? Am I right?
I was looking at the bulk operations and tried to play with the following code that somewhat mimics what I want to do:
```cpp
int main()
{
const std::size_t count = 1000;
std::vector output;
output.resize(count);
auto la = [&output](std::size_t index) {
// just something compute intensive
volatile double value=0;
for (std::size_t i=0; i<1'000'000; ++i){
value = std::sin(std::cos(std::sqrt(sum)));
}
output[index] = index % 2;
std::cout << index << " " << std::this_thread::get_id() << std::endl;
};
{
static_thread_pool ctx(4);
auto sched = ctx.get_scheduler();
sync_wait(bulk_join(
bulk_transform(
bulk_schedule(sched, count),
la, par_unseq)));
}
}
```
When running this using g++10.2 I am seeing all the work pretty much only executed by a single thread.
I am seeing very little work stealing. Is this expected? Or am i doing something wrong?
So based on the above result, and considering that the work load is pretty "static" (I know in advance that all operations are the same so I can just distribute chunks over threads to operate on) I tried to find a different solution to "force" using multiple threads.
```cpp
{
int chunk = count/4;
unifex::static_thread_pool ctx1(1);
auto sch1 = ctx1.get_scheduler();
auto w1 = for_each(on_stream(sch1, range_stream{0, chunk}), la );
unifex::static_thread_pool ctx2(1);
auto sch2 = ctx2.get_scheduler();
auto w2 = for_each(on_stream(sch2, range_stream{chunk, 2*chunk}), la );
unifex::static_thread_pool ctx3(1);
auto sch3 = ctx3.get_scheduler();
auto w3 = for_each(on_stream(sch3, range_stream{chunk*2, 3*chunk}), la );
unifex::static_thread_pool ctx4(1);
auto sch4 = ctx4.get_scheduler();
auto w4 = for_each(on_stream(sch4, range_stream{chunk*3, 4*chunk}), la );
sync_wait(when_all(w1, w2, w3, w4));
}
```
I know that using streams has some implications. But I think for my usecase it is fine, maybe not ideal.
However, the solution above does what I want: I get all four threads working as supposed to.
I was wondering if there is a better way to schedule multiple senders rather than the manual approach I am doing above?
Overall, i would like to kindly ask if there is something else in the library that is more appropriate for doing such thing?
thanks in advance!
Contributor guide
Assessment
This issue has not been assessed yet.