Set OpenMP CPU affinity by default
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 13.6k
- Forks
- 2.6k
- PR merge metrics
- No merged PRs in 30d
Description
In some of my tests (especially of the memory-hard formats on multi-socket/NUMA systems but not only there and not only of those), setting GOMP_CPU_AFFINITY to cover the full range of logical CPUs improves performance (sometimes a lot, e.g. by 77% in an scrypt benchmark I ran the other day). Maybe we should have john itself do that (early enough that threads are not started yet), except when that env var is already set (even if to an empty string), OMP_NUM_THREADS is set, or/and --fork is used.
For --fork in combination with OpenMP (where we already reduce the per-process thread count accordingly), we would need to use different CPU number ranges for the different processes. We may, but let's start with the simpler change first. (In fact, this need for special handling of --fork is a reason why I don't just configure GOMP_CPU_AFFINITY on my systems globally, so it's also a reason to have it built into john, where we can have it conditional.)
A drawback of all/any of these changes is that performance could suffer when the system is under significant load by something else running on it (tasks, kernel threads, interrupt handlers, etc.) that also uses strict CPU affinity. In such cases, performance would have been unoptimal anyway, but it could become worse yet. A workaround, desirable in such cases anyway, would be for the user to set GOMP_CPU_AFFINITY differently or/and to set OMP_NUM_THREADS such that competition for the CPUs used by the other load is avoided or reduced.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing john's early startup path and the existing OpenMP and --fork handling. Check how environment variables are read, then verify the change only sets GOMP_CPU_AFFINITY when the stated exceptions do not apply and benchmark the relevant memory-hard formats on a multi-socket system.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100