Disaggregated flash attention with the Neural Engine as the device, as the NVIDIA path already does

tests/backend/pass/flash_attention_disaggregated.vx runs fused attention on a remote NVIDIA device
from a laptop. The same program shape has never run with the Apple Neural Engine as the device. This
issue is to make it.

What already exists, on the NVIDIA side

The disaggregated GPU case is done and documented in that file's header. The compiler compiles the
region to sm_80 PTX and carries the image in the dispatch payload; a worker that does not recognise
the kernel loads it with cuModuleLoadData and launches it. Run from a laptop against an A100 80GB
over an SSH tunnel:

[Vx worker] TRANSFER 2048 bytes -> handle 1000000503000     (Q)
[Vx worker] TRANSFER 4096 bytes -> handle 1000000603800     (K)
[Vx worker] TRANSFER 4096 bytes -> handle 1000000704800     (V)
[Vx worker] TRANSFER 2048 bytes -> handle 1000000805800     (O)
[Vx worker] DISPATCH vx_npu_kernel_0 from its own image
[Vx worker] served 10 message(s): TRANSFER=4 (12384 B) DISPATCH=1 (11312 B) FREE=4 (32 B) FETCH=1 (16 B)

510 of 512 output elements identical to the same program run entirely on the host; two differ by
1.0e-5, which is the device's ex2.approx.ftz.f32 against the host's expf.

Why the Apple side is not the same job

The ANE runs CoreML models, not compiled kernels. The GPU path works because a kernel image can
be built and shipped in the payload. There is no equivalent for the Neural Engine: it executes a
compiled .mlmodelc, and the compiler does not produce one. So the "carry the kernel with the call"
mechanism does not transfer, and the worker-side story has to be different.

Flash attention is not a GEMM. The ANE route added in Vx#173 handles a classified square f16
matmul by looking up a prebuilt primitive. Fused attention exists precisely so that S = QKᵀ is
never materialised, so decomposing it into GEMMs defeats the algorithm -- the same objection Vx#319
records against routing FA through cuBLAS.

Why it is nonetheless tractable

Both halves of the missing piece already exist.

• The compiler already classifies fused attention. vx_attention_plan in
  runtime/vx_dispatch_plan.h, with role parsing, landed in Vx#378 for the vendor-kernel routing.
  An ANE route plugs into the same classification a GPU provider does.
• CoreML can express it as one operation. MIL has scaled_dot_product_attention, so an attention
  primitive is a model the existing generator can build, exactly as it now builds
  matmul_<d>x<d>_fp16.mlmodelc.

So the shape of the work is the shape of what Vx#173 just did for matmul, one op up.

Steps

• Generate a scaled_dot_product_attention fp16 primitive in
  scripts/generate_ane_primitives.py, parameterised by sequence length and head dimension the way
  the matmul primitives are parameterised by square size.
• Confirm with MLComputePlan that CoreML actually prefers the Neural Engine for it. Do this
  before writing any dispatch code. For matmul the answer was CPU for every fp32 size and for f16
  below 512; attention may have its own threshold, or may not be ANE-preferred at the shapes we care
  about, and that is a legitimate outcome to discover early rather than late.
• Route a classified vx_attention_plan to it in runtime/npu_dispatch.mm, alongside the
  existing GEMM route.
• Then the disaggregated half: decide what the worker loads. The model is a file rather than an
  image in the payload, so either the worker has the primitives already, or .mlmodelc becomes
  something the transport can carry. This is the genuinely new design question in the issue.
• A test in the shape of flash_attention_disaggregated.vx, with the answer compared against the
  host run the way that file compares 510 of 512 elements.

What would make it a real matrix row

Vx#435 grades the Apple cell T2 today on the strength of one square matmul. Fused attention running
on the ANE, disaggregated, would be the Apple counterpart of the NVIDIA row above -- and it is the
only thing that would let "one program, prefill here and decode there" be said about a second vendor.

Related: Vx#435 (the matrix), Vx#173 (the shape lookup this builds on), Vx#378 (the attention
classification), Vx#319 (why FA cannot be decomposed into GEMM calls), Vx#405 (the dtype gap that
makes an f32 attention on the ANE compile and silently run on the CPU).

Start with scripts/generate_ane_primitives.py and use MLComputePlan to determine whether CoreML prefers the Neural Engine for scaled dot-product attention. Then read vx_attention_plan in runtime/vx_dispatch_plan.h and the existing GEMM route in runtime/npu_dispatch.mm. Done includes an ANE attention primitive, dispatch handling, a worker transport decision, and a test shaped like tests/backend/pass/flash_attention_disaggregated.vx.