QNN parallel performance issues

@cccclai @haowhsu-quic I think the performance issues https://github.com/pytorch/executorch/issues/12537 that have been reported here before are basically located here. so I plan to open a new issue here.

🐛 Describe the bug

The performance of the QNN backend conv2d kernel (1,5) and input hw (1,200) is twice as bad as the performance of the kernel (5,1) and input hw (200,1). This difference shouldn't be this significant. There's something wrong with the parallelization strategy.
The reason for this experiment is that I found that there are two options when converting conv1d to conv2d. Logically, the performance should be similar, but this is not the case with QNN. In some cases, nch1 performs better, and sometimes nc1w performs better.
Two models

class LargeConv2d5x1Sequential(nn.Module):
    def __init__(self, bias=True):
        super().__init__()
        self.net = nn.Sequential(
            nn.Conv2d(64, 128, kernel_size=(5,1), padding=(2,0), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(128, 256, kernel_size=(5,1), padding=(2,0), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(256, 512, kernel_size=(5,1), padding=(2,0), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(512, 1024, kernel_size=(5,1), padding=(2,0), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(1024, 512, kernel_size=(5,1), padding=(2,0), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(512, 256, kernel_size=(5,1), padding=(2,0), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(256, 128, kernel_size=(5,1), padding=(2,0), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(128, 64, kernel_size=(5,1), padding=(2,0), bias=bias),
        )

    def forward(self, x):
        return self.net(x)

class LargeConv2d1x5Sequential(nn.Module):
    def __init__(self, bias=True):
        super().__init__()
        self.net = nn.Sequential(
            nn.Conv2d(64, 128, kernel_size=(1,5), padding=(0,2), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(128, 256, kernel_size=(1,5), padding=(0,2), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(256, 512, kernel_size=(1,5), padding=(0,2), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(512, 1024, kernel_size=(1,5), padding=(0,2), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(1024, 512, kernel_size=(1,5), padding=(0,2), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(512, 256, kernel_size=(1,5), padding=(0,2), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(256, 128, kernel_size=(1,5), padding=(0,2), bias=bias),
            nn.LeakyReLU(0.1),
            nn.Conv2d(128, 64, kernel_size=(1,5), padding=(0,2), bias=bias),
        )

    def forward(self, x):
        return self.net(x)

The performance is as follows

Versions

Collecting environment information...
PyTorch version: 2.9.0.dev20250725+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.5 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: 14.0.0-1ubuntu1.1
CMake version: version 3.31.6
Libc version: glibc-2.35

Python version: 3.10.0 (default, Mar  3 2022, 09:58:08) [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-6.6.47-12.tl4.x86_64-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Address sizes:                        46 bits physical, 48 bits virtual
Byte Order:                           Little Endian
CPU(s):                               16
On-line CPU(s) list:                  0-15
Vendor ID:                            GenuineIntel
BIOS Vendor ID:                       Smdbmds
Model name:                           Intel(R) Xeon(R) Gold 6133 CPU @ 2.50GHz
BIOS Model name:                      3.0
CPU family:                           6
Model:                                94
Thread(s) per core:                   1
Core(s) per socket:                   16
Socket(s):                            1
Stepping:                             3
BogoMIPS:                             4988.28
Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 arat
Hypervisor vendor:                    KVM
Virtualization type:                  full
L1d cache:                            512 KiB (16 instances)
L1i cache:                            512 KiB (16 instances)
L2 cache:                             64 MiB (16 instances)
L3 cache:                             27.5 MiB (1 instance)
NUMA node(s):                         1
NUMA node0 CPU(s):                    0-15
Vulnerability Gather data sampling:   Unknown: Dependent on hypervisor status
Vulnerability Itlb multihit:          KVM: Mitigation: VMX unsupported
Vulnerability L1tf:                   Mitigation; PTE Inversion
Vulnerability Mds:                    Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
Vulnerability Meltdown:               Mitigation; PTI
Vulnerability Mmio stale data:        Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed:               Vulnerable
Vulnerability Spec rstack overflow:   Not affected
Vulnerability Spec store bypass:      Vulnerable
Vulnerability Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:             Mitigation; Retpolines; STIBP disabled; RSB filling; PBRSB-eIBRS Not affected; BHI Retpoline
Vulnerability Srbds:                  Unknown: Dependent on hypervisor status
Vulnerability Tsx async abort:        Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown

Versions of relevant libraries:
[pip3] executorch==0.8.0a0+2c84f70
[pip3] numpy==2.2.6
[pip3] nvidia-cublas-cu12==12.6.4.1
[pip3] nvidia-cuda-cupti-cu12==12.6.80
[pip3] nvidia-cuda-nvrtc-cu12==12.6.77
[pip3] nvidia-cuda-runtime-cu12==12.6.77
[pip3] nvidia-cudnn-cu12==9.5.1.17
[pip3] nvidia-cufft-cu12==11.3.0.4
[pip3] nvidia-curand-cu12==10.3.7.77
[pip3] nvidia-cusolver-cu12==11.7.1.2
[pip3] nvidia-cusparse-cu12==12.5.4.2
[pip3] nvidia-cusparselt-cu12==0.6.3
[pip3] nvidia-nccl-cu12==2.26.2
[pip3] nvidia-nvjitlink-cu12==12.6.85
[pip3] nvidia-nvtx-cu12==12.6.77
[pip3] onnx==1.18.0
[pip3] onnxruntime==1.22.0
[pip3] pytorch_tokenizers==0.1.0
[pip3] torch==2.9.0.dev20250725+cpu
[pip3] torchao==0.13.0+git2eb4f9762
[pip3] torchaudio==2.8.0.dev20250725+cpu
[pip3] torchdata==0.11.0
[pip3] torchsr==1.0.4
[pip3] torchtune==0.6.1
[pip3] torchvision==0.24.0.dev20250725+cpu
[pip3] triton==3.3.1
[conda] executorch                0.8.0a0+2c84f70          pypi_0    pypi
[conda] numpy                     2.2.6                    pypi_0    pypi
[conda] nvidia-cublas-cu12        12.6.4.1                 pypi_0    pypi
[conda] nvidia-cuda-cupti-cu12    12.6.80                  pypi_0    pypi
[conda] nvidia-cuda-nvrtc-cu12    12.6.77                  pypi_0    pypi
[conda] nvidia-cuda-runtime-cu12  12.6.77                  pypi_0    pypi
[conda] nvidia-cudnn-cu12         9.5.1.17                 pypi_0    pypi
[conda] nvidia-cufft-cu12         11.3.0.4                 pypi_0    pypi
[conda] nvidia-curand-cu12        10.3.7.77                pypi_0    pypi
[conda] nvidia-cusolver-cu12      11.7.1.2                 pypi_0    pypi
[conda] nvidia-cusparse-cu12      12.5.4.2                 pypi_0    pypi
[conda] nvidia-cusparselt-cu12    0.6.3                    pypi_0    pypi
[conda] nvidia-nccl-cu12          2.26.2                   pypi_0    pypi
[conda] nvidia-nvjitlink-cu12     12.6.85                  pypi_0    pypi
[conda] nvidia-nvtx-cu12          12.6.77                  pypi_0    pypi
[conda] pytorch-tokenizers        0.1.0                    pypi_0    pypi
[conda] torch                     2.9.0.dev20250725+cpu          pypi_0    pypi
[conda] torchao                   0.13.0+git2eb4f9762          pypi_0    pypi
[conda] torchaudio                2.8.0.dev20250725+cpu          pypi_0    pypi
[conda] torchdata                 0.11.0                   pypi_0    pypi
[conda] torchsr                   1.0.4                    pypi_0    pypi
[conda] torchtune                 0.6.1                    pypi_0    pypi
[conda] torchvision               0.24.0.dev20250725+cpu          pypi_0    pypi
[conda] triton                    3.3.1                    pypi_0    pypi

cc @cccclai @cbilgin

Start by reading the related issue #12537 and reproducing the two LargeConv2d5x1Sequential and LargeConv2d1x5Sequential cases on the QNN backend. Compare their timings and trace the QNN conv2d kernel's parallelization path; done means the unexpected performance gap is explained and the affected configuration no longer shows the reported regression.