large core count dev machines spawn too many threads
- Dominant language
- C++
- Stars
- 20.5k
- Forks
- 2.3k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 74
Description
⚠️ **Issues not using this template will be systematically closed.**
**Describe the bug**
Development machines with large core count spawn too many threads.
`std::thread::hardware_concurrency()` may return the hardware thread count of available hardware threads. In general this is not handled in the most ideal way to prevent excessive resource consumption. This will impact debugging performance on a host machine
Sample source tree grep for `std::thread::hardware_concurrency()`:
```
filament/src/details/Engine.cpp: int threadCount = (int)std::thread::hardware_concurrency() - 2;
filament/src/details/Engine.cpp: uint32_t const id = std::thread::hardware_concurrency() - 1;
third_party/tinyexr/tinyexr.h: int num_threads = std::max(1, int(std::thread::hardware_concurrency()));
third_party/tinyexr/tinyexr.h: int num_threads = std::max(1, int(std::thread::hardware_concurrency()));
third_party/basisu/encoder/basisu_comp.cpp: num_threads = basisu::maximum(1, std::thread::hardware_concurrency());
third_party/basisu/encoder/basisu_frontend.cpp: max_threads = m_params.m_multithreaded ? minimum(std::thread::hardware_concurrency(), cMaxCodebookCreationThreads) : 0;
third_party/basisu/encoder/basisu_frontend.cpp: max_threads = m_params.m_multithreaded ? minimum(std::thread::hardware_concurrency(), cMaxCodebookCreationThreads) : 0;
third_party/basisu/basisu_tool.cpp: // We use std::thread::hardware_concurrency() as a hint to determine the default # of threads to put into a pool.
third_party/basisu/basisu_tool.cpp: num_threads = std::thread::hardware_concurrency();
third_party/meshoptimizer/gltf/basisenc.cpp: uint32_t num_threads = settings.texture_jobs == 0 ? std::thread::hardware_concurrency() : settings.texture_jobs;
```
On my dev box it has 32 hardware threads. That means the engine is spawning 30 threads.
**To Reproduce**
Steps to reproduce the behavior:
1. Run any filament example of a high thread count system
**Expected behavior**
Not to impact debugging performance or consume excessive resources.
**Logs**
If applicable, copy **full** logs from your console here. Please *do not*
use screenshots of logs, copy them as text, use gist or attach an *uncompressed* file.
**Desktop (please complete the following information):**
- OS: Linux
- GPU: AMD Radeon
- Backend: Vulkan
**Smartphone (please complete the following information):**
- Device: [e.g. Pixel 2]
- OS: [e.g. Android Pie 9.0]
**Additional context**
A proposed solution might be:
```
static inline size_t get_platform_thread_count(size_t min_required_thread_count, size_t max_useful_thread_count) {
size_t count = std::thread::hardware_concurrency()
if (count <= min_required_thread_count) {
return min_required_thread_count;
}
else if(count <= max_useful_thread_count) {
return count;
}
else {
return max_useful_thread_count;
}
}
```
Contributor guide
Assessment
This issue has not been assessed yet.