refactor(packing): replace global HF monkeypatch with typed metadata

Context

AutoModel NEAT packing currently encodes both document boundaries and padding in an integer attention_mask such as [1, 1, 2, 2, 0]. The FlashAttention path then globally monkeypatches private Transformers functions, including _get_unpad_data and model-local create_causal_mask references, to recover per-document cu_seqlens.

PR NVIDIA-NeMo/Automodel#2996 hardens this legacy contract for Transformers 5.x and makes incompatible private APIs fail closed. That is the right compatibility fix for the current design, but it does not remove the process-wide private-API coupling.

Pinned Transformers exposes typed FlashAttention packing kwargs (cu_seq_lens_q, cu_seq_lens_k, max_length_q, and max_length_k). AutoModel should move packing metadata through a supported Transformers path or a typed AutoModel/HF adapter.

Related: NVIDIA-NeMo/Automodel#2996, NVIDIA-NeMo/Automodel#2985.

Proposed scope

• Introduce typed packed-sequence metadata produced by the LLM and VLM NEAT collaters.
• Separate document IDs from the normal padding attention_mask contract.
• Thread explicit cumulative sequence lengths and maximum lengths through supported HF FlashAttention kwargs or a typed attention adapter.
• Preserve per-document IDs for AutoModel consumers such as SqrtCrossEntropy, Qwen3.5 MTP, and hybrid/linear-attention paths without overloading the HF padding mask.
• Define backend capability checks explicitly: support FA2 first; enable FA3 only when AutoModel can select and validate it.
• Remove global mutation of private Transformers functions once parity coverage is in place.

Acceptance criteria

• No global monkeypatch of Transformers _preprocess_mask_arguments, _get_unpad_data, or model-module create_causal_mask for NEAT packing.
• No cross-document attention for padded and unpadded packs.
• Model-level parity coverage for LLM and VLM packing, including batch size greater than one, uneven padding, image/video inputs, and Qwen mRoPE position IDs.
• Existing SDPA/eager block-causal packing behavior remains unchanged.
• Loss and auxiliary model paths that consume packed document IDs continue to receive equivalent typed metadata.
• Failure is explicit when a model or attention backend cannot consume the typed packing contract.

Start by reviewing the LLM and VLM NEAT collaters, the private Transformers functions listed in the issue, and related PRs #2996 and #2985. Trace how packed metadata reaches FlashAttention and consumers such as SqrtCrossEntropy and Qwen3.5 MTP; done means typed metadata replaces the monkeypatch while the stated parity and failure checks pass.