AcademySoftwareFoundation / AcademySoftwareFoundation/OpenImageIO

[Enhancement] Use application provided ThreadPool

Open
#1,763 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.4k
Forks
698
Avg merge
3d 9h
Merged PRs (30d)
48

Description

When an external application already has its own threading technology (be it a thread-pool or w/e), it is hard to wire it to OpenImageIO's own thread-pool.

If we enable multi-threading with the oiio threadpool we get overthreading because a threadpool is a centralized way of managing threads workload.

If we disable multi-threading for OIIO using oiio-threads, we then get no multi-threading for things like parallel_convert_image.

OIIO being a library, it should be able to operate with another application's thread-pool, so that we get the best performance possible out of OIIO.

I tried to hack around and modify the thread-pool API so it could be inherited by the application and we could replace the default thread-pool by forwarding calls to our own threading mechanism, but the API of the thread-pool being complicated with templated functors makes it impossible to achieve.
The other way around would be to use OIIO's thread-pool as main threading backend of the application but we do not want that as there is better suited threading technologies available.

I did the same with OpenEXR's thread-pool and it works well because their thread-pool API is essentially very simple with just a "addTask" function and their simple Task and TaskGroup classes.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing OpenImageIO's thread-pool API and the parallel_convert_image entry point, then compare the simpler OpenEXR addTask, Task, and TaskGroup model described in the issue. Done means an external application can supply or integrate its thread pool so OIIO's parallel operations avoid overthreading while retaining multithreading.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend-api-design, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.