OpenMathLib / OpenMathLib/OpenBLAS
Parallelization along K-dimension (Parallel Reduction) for GEMM with small M/N and large K
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 7.6k
- Forks
- 1.7k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 42
Description
Hi OpenBLAS team,
I noticed that zgemm (and other GEMM functions) falls back to single-threaded execution when M and N are small (e.g., 32) but K is extremely large (e.g., 1,000,000).
On my many-core system, this leaves most cores idle. Given the large K size, parallelizing the K-loop (via parallel reduction) should theoretically offer significant speedup. I perform the matrix partitioning (of k) externally, and then use multithreading to call zgemm, but the performance is only average.
Questions:
Does OpenBLAS currently support threading along the K-dimension for this shape?
If not, are there any plans to implement parallel reduction for large K?
My Machine Info:
Thanks!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with zgemm's current threading path and the GEMM K-loop, using the reported small-M/N, large-K shape to reproduce the single-threaded behavior. Check whether K-dimension parallel reduction is already supported; done would require a clear implementation decision and performance evidence for this case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100