Implement tail-folded vector epilogue loop
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
Enable tail-folding on vectorized epilogue, so that we can have:
- Vectorized main loop
- Tail-folded vectorized epilogue loop.
Motivation behind this work:
The current vectorization styles force either tail-folding on the main vector loop with no interleaving, or unpredicated main vector loop with interleaving.
The first style prevents us from getting the benefit of high interleaving when it’s beneficial/possible, and the second one prevents tail-folding while it could be beneficial specially for low trip count.
The proposed hybrid approach of having unpredicated main vector loop with tail-folded vector epilogue combines the strengths of both styles, apply high interleaving in the main loop where most iterations get executed unconditionally and also predicate the epilogue loop where tail-folding is beneficial for low-trip count(low remaining iterations), and in some cases, the epilogue could be a single predicated iteration.
Right now this work is intended to be enabled by a flag, in the future there should be a dynamic selection between different vectorization styles according to the costs/profitability.
Contributor guide
Assessment
This issue has not been assessed yet.