AssertionError: Mul tiheadAttention does not support NestedTensor outside of its fast path

### 🐛 Describe the bug

- [ ] NestedTensor outside of its fast path

Cases:
op_ut,third_party.torch-xpu-ops.test.xpu.test_nn_xpu.TestNNDeviceTypeXPU,test_transformerencoderlayer_xpu_float16
op_ut,third_party.torch-xpu-ops.test.xpu.test_nn_xpu.TestNNDeviceTypeXPU,test_transformerencoderlayer_xpu_float32
op_ut,third_party.torch-xpu-ops.test.xpu.test_transformers_xpu.TestTransformersXPU ,test_with_nested_tensor_input_xpu

| op_ut | third_party.torch-xpu-ops.test.xpu.test_transformers_xpu.TestTransformersXPU | test_with_nested_tensor_input_xpu | failed | AssertionError: Mul
tiheadAttention does not support NestedTensor outside of its fast path. The fast path was not hit because some Tensor argument's device is neither one of
cpu, cuda or privateuseone | XML |

```
transformer_encoder = nn.TransformerEncoder(encoder_layer, num_layers=3, enable_nested_tensor=True).to(device)
...
out = transformer_encoder(src=x, src_key_padding_mask=None)
```
The first statement transferred the tensor to xpu:0, while the 2nd statement got assertion and when debugging the device of TransformerEncoderLayer is None. So I think there could be some problem with nested tensor .to(device) operation. Need further investigation to confirm.

```
======================================================== FAILURES ========================================================
_________________________________ TestTransformersXPU.test_with_nested_tensor_input_xpu __________________________________
Traceback (most recent call last):
File "/home/daisyden/upstream/test_transformers/third_party/ci/test/xpu/../../../../test/test_transformers.py", line 955, in test_wi
th_nested_tensor_input
out = transformer_encoder(src=x, src_key_padding_mask=None)
File "/home/daisyden/miniforge3/envs/test_transformers/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1784, in _wrap
ped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/daisyden/miniforge3/envs/test_transformers/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1795, in _call
_impl
return forward_call(*args, **kwargs)
File "/home/daisyden/miniforge3/envs/test_transformers/lib/python3.10/site-packages/torch/nn/modules/transformer.py", line 538, in f
orward
output = mod(
File "/home/daisyden/miniforge3/envs/test_transformers/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1784, in _wrap
ped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/daisyden/miniforge3/envs/test_transformers/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1795, in _call
_impl
return forward_call(*args, **kwargs)
File "/home/daisyden/miniforge3/envs/test_transformers/lib/python3.10/site-packages/torch/nn/modules/transformer.py", line 951, in f
orward
+ self._sa_block(x, src_mask, src_key_padding_mask, is_causal=is_causal)
File "/home/daisyden/miniforge3/envs/test_transformers/lib/python3.10/site-packages/torch/nn/modules/transformer.py", line 965, in _
sa_block
x = self.self_attn(
File "/home/daisyden/miniforge3/envs/test_transformers/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1784, in _wrap
ped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/daisyden/miniforge3/envs/test_transformers/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1795, in _call
_impl
return forward_call(*args, **kwargs)
File "/home/daisyden/miniforge3/envs/test_transformers/lib/python3.10/site-packages/torch/nn/modules/activation.py", line 1449, in f
orward
assert not any_nested, (
AssertionError: MultiheadAttention does not support NestedTensor outside of its fast path. The fast path was not hit because some Tens
or argument's device is neither one of cpu, cuda or privateuseone

To execute this test, run the following from the base repo dir:
PYTORCH_TEST_WITH_SLOW=1 python ../../test/test_transformers.py TestTransformersXPU.test_with_nested_tensor_input_xpu

```

### Versions

pytorch-triton-xpu 3.5.0+git1b0418a9
torch 2.10.0.dev20251012+xpu
torchaudio 2.8.0.dev20251012+xpu
torchvision 0.25.0.dev20251012+xpu