[flang][x86-64] masked vectorization on cold data + sparse mask can significantly slow down code

This is related to [[RFC][Flang] Improving temporary and local array allocations](https://discourse.llvm.org/t/rfc-flang-improving-temporary-and-local-array-allocations-unify-stackarray-memoryallocation/91353/3?u=jeanperier) where I noticed a 100% slowdown (124s to 255s) with flang on WRF from SPEC 2006 running on Zen4 when putting Fortran automatic arrays on the heap (instead of on the stack as they currently are).

This slowdown reproduced on different X86-64 architectures (50% slowdown on Emerald Rapids), but not on aarch64 (no measurable performance difference).

After analysis, the issue is that WRF 2006 `sint` kernel that contains masked operations is vectorized and that its mask is quite sparse (10-20% of true value). The kernel is touching 11 arrays including the mask. 9 of these arrays are Fortran automatic arrays, and moving them to the heap causes them to be cold in cache/TLB when hitting the kernel (when using the stack, the stack storage is a lot more likely to be hot from previous stack frame usage, which is the case in the benchmark).

The masked vectorization causes the cache/TLB miss for all these arrays all the time drastically increasing waiting times over non vectorized code.

This is confirmed by compiling SINT.F90 with `-fno-vectorize` which allows reaching the same performance than with the stack (warm addresses) the kernel itself is 10 to 20 times slower with vectorization.

The micro benchmark [masked_vector_bench.tar.gz](https://github.com/user-attachments/files/30457029/masked_vector_bench.tar.gz) shows how cold ALLOCATE are much slower than warm stack on the same kernel. The only aspect from WRF that was kept in the kernel is the number of arrays being touched and the mask density of true value, the operations themselves are irrelevant.

It also shows that flang is the only compiler that is impacted that hard by cold vs warm data on this kernel, with the exception of ifx which is still 4 times faster in the end (which is consistent with the fact that ifx suffers a 30% slowdown only when using -heap-arrays).

I am opening this issue to discuss what we can and should do about this given I was proposing to move automatic arrays on the heap by default (except with -fstack-arrays/-Ofast).

Is there a way to improve vectorization of such masked kernels so that they are not hit by cold data as hard as they currently are? Could some option be added to not do masked vectorization when more than a certain number of base addresses are touched?

Start with the WRF 2006 SINT.F90 kernel and reproduce the comparison using -fno-vectorize, then inspect the supplied masked_vector_bench.tar.gz microbenchmark. Compare cold allocated arrays with warm stack data across the reported x86-64 cases; done means identifying a justified compiler-side direction and confirming its performance impact without regressing the relevant masked-vectorization behavior.

Written by the indexing model from the issue text.