[FEAT] Wan 2.2: fuse NVFP4 quantization with preceding LayerNorm/AdaLN and GELU-tanh

Description

The Wan 2.2 T2V DiT (tensorrt_llm/_torch/visual_gen/models/wan/transformer_wan.py)
runs each of its 34 unignored transformer blocks through the following hot
path on NVFP4 inference:

input -> LayerNorm -> (AdaLN modulation: *(1+scale_msa) + shift_msa) ->
         cast to fp32 -> fp4_quantize (per-block amax + e2m1x2 pack) ->
         NVFP4 GEMM

Each LayerNorm / activation feeding into an NVFP4 GEMM dispatches three
distinct CUDA kernels (norm or activation, cast, fp4_quantize) before
the GEMM can launch. On a B200, this constitutes a measurable
launch-and-memory-bandwidth overhead.

The same pattern appears in the MLP path:

hidden -> up_proj -> GELU-tanh -> fp4_quantize -> down_proj

Proposed approach

Mirror the kernel-fusion strategy from PR
#11473
(Mamba2/Nemotron-H NVFP4 layernorm + activation fusion):

1. Add a fusedLayerNormQuant CUDA kernel that performs LayerNorm
   (optionally with affine weight/bias and AdaLN modulation) and emits
   NVFP4-packed activations + swizzled per-16-element FP8 (e4m3) scale
   factors directly in the layout the CUTLASS NVFP4 GEMM kernels expect.

2. Extend the existing fusedActivationQuant family with a
   gelu_tanh + NVFP4 variant for the MLP down_proj input.

3. Gate both fused paths on the presence of a per-tensor calibrated
   input_scale (static NVFP4) and the layer being on the unignored
   list from the checkpoint's quantization_config. Fall back to the
   unfused kernels otherwise.

This is purely an inference-time optimization: numerics must be within
cosine-similarity ≥ 0.995 of the unfused path on a full forward pass.

Motivation

Calibrated checkpoint already exists:
nvidia/Wan2.2-T2V-A14B-Diffusers-NVFP4.

Preliminary B200 numbers (n=20, after warmup, 480p latent / 1560 tokens
/ 77-token text condition):

nsys NVTX breakdown attributes the win to fewer kernel launches per
norm/activation site.

Scope and risks

• Touches one new kernel directory (fusedLayerNormQuant/) and extends
  fusedActivationQuant. No public-API changes.
• Behind a kill switch (TRTLLM_DISABLE_NVFP4_LAYERNORM_FUSION=1) for
  safe A-B.
• Inactive without a calibrated NVFP4 checkpoint; falls back to today's
  unfused path.
• Validated on a B200; should be compatible with all SM100+ targets.

Related PRs

• #11473 — reference implementation for Mamba2/Nemotron-H NVFP4 layernorm
  • activation fusion (the design this PR adapts).

Proposed PR

A PR implementing this is ready on a fork:
https://github.com/anikaj-eng/TensorRT-LLM/tree/feat/wan22-nvfp4-fusion
(will be opened against main once this issue is reviewed.)