[Performance] CPU EP does not fuse float16 Swish/SiLU to QuickGelu (slow on ARM)

@tianleiwu is already working on this.

Since Jun 16, 2026.

### Describe the issue

On the CPU execution provider, the Swish/SiLU activation `x * Sigmoid(x)` is fused into the fused `QuickGelu` kernel only for `float`. On a `float16` graph the activation runs as separate `Sigmoid` + `Mul` ops, which is meaningfully slower on ARMv8.2-A CPUs (Raspberry Pi 5 / Cortex-A76). Weights and MatMul/Conv are fine in fp16; only the activation regresses.

This is separate from #25824 / #25913 (which enabled fp16 `MatMul`/`Gemm` on CPU). I tested 1.25.1 and 1.26.0 and the activation behavior is the same; that work did not cover the activation fusion.

### Verified against source (checked locally on rel-1.25.1; QuickGelu also confirmed on main)

- `core/providers/cpu/cpu_execution_provider.cc` registers `Sigmoid` on the CPU EP for `float` and `double` only. There is no `MLFloat16` `Sigmoid` CPU kernel, so an fp16 `Sigmoid` on CPU must be cast-wrapped (fp16 -> fp32 -> fp16).
- `contrib_ops/cpu/activations.cc` registers `QuickGelu` for `float` only (`QuickGelu`, kMSDomain, kCpuExecutionProvider). No `MLFloat16` registration.
- `core/optimizer/quick_gelu_fusion.cc` matches `x * Sigmoid(alpha*x)` and fuses to `QuickGelu`. It is EP-gated (CPU is in the target list) but not data-type-gated, and it matches a contiguous `Sigmoid -> Mul`; it does not match through intervening `Cast` nodes.
- `core/mlas/lib/activate_fp16.cpp` / `MlasFp16Activation` (`mlas.h`) already provide fp16 activation routines, so a CPU fp16 kernel has an existing MLAS backing.

Net effect on a CPU fp16 graph: the Swish runs as cast-wrapped `Sigmoid` + `Mul` and is not fused, and there is no fp16 `QuickGelu` CPU kernel to fall back to.

### Measured impact

BirdNET v2.4 (CNN, EfficientNet-style backbone), single 3s @ 48 kHz segment, RPi5 Cortex-A76 x4, CPU EP, governor=performance, 150 iterations, median latency:

| Threads | fp32 | fp16 (all) | fp16, Swish kept fp32 |
|--------:|-----:|-----------:|----------------------:|
| 1 | 145 ms | 162 ms | 140 ms |
| 2 | 83 ms | 104 ms | 85 ms |
| 4 | 58 ms | 79 ms | 64 ms |

Keeping just the Swish `Sigmoid`/`Mul` in fp32 (so the existing fp32 `QuickGelu` fusion fires) while conv/matmul weights stay fp16 recovers ~20% at 4 threads and brings fp16 close to fp32. That isolates the activation fusion as the cause.

Op-level profile (single inference, fp16 model, CPU EP):

```
fp32 model: MatMul 288 ms | QuickGelu 27.7 ms (fused) | Conv ~100 ms | Cast ~0
fp16 model: MatMul 285 ms | Sigmoid 22 ms + Mul 95.6 ms (UNFUSED) | NhwcFusedConv 58 ms | Cast 14.8 ms (2.5%)
```

The `Cast` overhead is small (~2.5%), and `MatMul` is precision-neutral on A76 (it lacks `asimdfhm`/FMLAL, so fp16 GEMM widens to fp32). The whole gap is the unfused Swish: ~118 ms of `Sigmoid`+`Mul` vs ~28 ms of fused `QuickGelu`.

x86 note: on a CPU without native fp16 (i7-1260P, avx2+f16c, no AVX512-FP16) fp16 is within ~4% of fp32 and the activation fusion barely matters, because elementwise throughput is high. The regression is ARM-specific.

### Question for maintainers

Since the fusion pass is not dtype-gated, what prevents the fp16 Swish from fusing on CPU in practice? My working assumption is that precision `Cast` nodes get inserted around the fp16 activation (which has no native CPU kernel) and break the contiguous `Sigmoid -> Mul` the pass requires, and that even if it fused there is no fp16 `QuickGelu` CPU kernel to target. Could you confirm the exact interaction?

### Proposed fix

Register an `MLFloat16` `QuickGelu` kernel (and any fp16 elementwise activation kernel needed so casts are not inserted ahead of the fusion) on the CPU EP, with a graceful fp32 fallback where the CPU lacks native fp16 (so x86 is unaffected). CUDA EP already registers `QuickGelu` for `MLFloat16`, so the fp16 path works there.

This mainly benefits ARM64 edge/mobile inference, where fp16 is attractive for model size / memory.

### Willing to contribute

Yes. I am happy to implement the kernel registration plus optimizer and kernel tests, and provide the before/after numbers above, pending agreement on the approach.

### Urgency

Not urgent. There is a workaround (keep the activation in fp32 during conversion), but it gives back some of the fp16 memory benefit.