[release/2.13] Test_xpu.py: Fatal Python error: Aborted on windows

### 🐛 Describe the bug

The test file: https://github.com/pytorch/pytorch/blob/main/test/test_xpu.py has below issue in windows CI.
Detail log: https://github.com/intel/torch-xpu-ops/actions/runs/24223088962/job/70724781107
````
00007FF80CBB520800007FF80CBABC10 torch_python.dll!torch::autograd::THPCppFunction_requires_grad [ @ ]
00007FF8CB6619D700007FF8CB6619C0 ucrtbase.dll!terminate [ @ ]
00007FF8AD1B1943 VCRUNTIME140_1.dll! [ @ ]
00007FF8AD1B21AF VCRUNTIME140_1.dll! [ @ ]
00007FF8AD1B220D VCRUNTIME140_1.dll! [ @ ]
00007FF8AD1B408200007FF8AD1B3FD0 VCRUNTIME140_1.dll!_CxxFrameHandler4 [ @ ]
00007FF8CDEA61CF00007FF8CDEA6130 ntdll.dll!_chkstk [ @ ]
00007FF8CDD523A700007FF8CDD51E10 ntdll.dll!RtlLocateExtendedFeature [ @ ]
00007FF8CDD4A96100007FF8CDD4A740 ntdll.dll!RtlRaiseException [ @ ]
00007FF8CB2673FA00007FF8CB267370 KERNELBASE.dll!RaiseException [ @ ]
00007FF8B8F752C700007FF8B8F75230 VCRUNTIME140.dll!CxxThrowException [ @ ]
00007FF8476890B600007FF8476878C0 sycl8.dll!sycl::_V1::queue::ext_codeplay_supports_fusion [ @ ]
00007FF84784031100007FF847815570 sycl8.dll!_sycl_register_lib [ @ ]
00007FF89EFECAD700007FF89EFECA10 c10_xpu.dll!c10::xpu::syncStreamsOnDevice [ @ ]
00007FF89EFE6D0900007FF89EFE63D0 c10_xpu.dll!c10::xpu::XPUCachingAllocator::releasePool [ @ ]
00007FF89EFE046400007FF89EFE03C0 c10_xpu.dll!c10::xpu::MemPool::device [ @ ]
00007FF89EFDD5C400007FF89EFDD540 c10_xpu.dll!c10::xpu::MemPool::~MemPool [ @ ]
00007FF80CB78C0D00007FF80C8B0460 torch_python.dll!torch::PythonArgs::toSymIntOptional [ @ ]
00007FF80CB78D4600007FF80C8B0460 torch_python.dll!torch::PythonArgs::toSymIntOptional [ @ ]
00007FF80C7B1CCD00007FF80C7B08A0 torch_python.dll!c10::ivalue::Await::args [ @ ]
00007FF80C7C183B00007FF80C7C13A0 torch_python.dll!c10::ivalue::Future::waitAndThrow [ @ ]
00007FF85B24229700007FF85B241B80 python310.dll!PyType_GenericNew [ @ ]
00007FF85B2E67EE00007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FF85B2E99C900007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FF85B1DE73E00007FF85B1DE700 python310.dll!PyFunction_Vectorcall [ @ ]
00007FF85B1E0A9100007FF85B1E06A0 python310.dll!PyCell_Set [ @ ]
00007FF85B1E0C8600007FF85B1E0B90 python310.dll!PyMethod_Self [ @ ]
00007FF85B2EB5B400007FF85B2EAFB0 python310.dll!PyEval_GetFuncDesc [ @ ]
00007FF85B2E654C00007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FF85B2E99C900007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FF85B1DE73E00007FF85B1DE700 python310.dll!PyFunction_Vectorcall [ @ ]
00007FF85B1E0A9100007FF85B1E06A0 python310.dll!PyCell_Set [ @ ]
00007FF85B1E0C4D00007FF85B1E0B90 python310.dll!PyMethod_Self [ @ ]
00007FF85B2E2A9100007FF85B2E2950 python310.dll!PyOS_URandomNonblock [ @ ]
00007FF85B2EB3A200007FF85B2EAFB0 python310.dll!PyEval_GetFuncDesc [ @ ]
00007FF85B2E71A400007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
Fatal Python error: Aborted

Thread 0x00005d9c (most recent call first):

Thread 0x000049e0 (most recent call first):
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\threading.py", line 324 in wait
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\threading.py", line 607 in wait
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\tqdm\_monitor.py", line 60 in run
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\threading.py", line 1016 in _bootstrap_inner
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\threading.py", line 973 in _bootstrap

Thread 0x00004e20 (most recent call first):
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\socket.py", line 293 in accept
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\pytest_rerunfailures.py", line 433 in run_server
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\threading.py", line 953 in run
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\threading.py", line 1016 in _bootstrap_inner
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\threading.py", line 973 in _bootstrap

Current thread 0x000065f8 (most recent call first):
File "C:\actions-runner\_work\torch-xpu-ops\pytorch\test\test_xpu.py", line 3050 in test_mempool_id
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\torch\testing\_internal\common_utils.py", line 3444 in wrapper
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\unittest\case.py", line 549 in _callTestMethod
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\unittest\case.py", line 591 in run
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\torch\testing\_internal\common_utils.py", line 3596 in _run_custom
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\torch\testing\_internal\common_utils.py", line 3627 in run
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\unittest\case.py", line 650 in __call__
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\_pytest\unittest.py", line 333 in runtest
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\_pytest\runner.py", line 169 in pytest_runtest_call
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\pluggy\_callers.py", line 121 in _multicall
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\pluggy\_manager.py", line 120 in _hookexec
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\pluggy\_hooks.py", line 512 in __call__
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\_pytest\runner.py", line 262 in
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\_pytest\runner.py", line 341 in from_call
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\_pytest\runner.py", line 261 in call_runtest_hook
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\_pytest\runner.py", line 222 in call_and_report
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\_pytest\runner.py", line 133 in runtestprotocol
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\_pytest\runner.py", line 114 in pytest_runtest_protocol
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\pluggy\_callers.py", line 121 in _multicall
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\pluggy\_manager.py", line 120 in _hookexec
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\pluggy\_hooks.py", line 512 in __call__
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\_pytest\main.py", line 348 in pytest_runtestloop
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\pluggy\_callers.py", line 121 in _multicall
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\pluggy\_manager.py", line 120 in _hookexec
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\pluggy\_hooks.py", line 512 in __call__
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\_pytest\main.py", line 323 in _main
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\_pytest\main.py", line 269 in wrap_session
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\_pytest\main.py", line 316 in pytest_cmdline_main
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\pluggy\_callers.py", line 121 in _multicall
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\pluggy\_manager.py", line 120 in _hookexec
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\pluggy\_hooks.py", line 512 in __call__
File "C:\ProgramData\miniforge3\envs\windows_ut\lib\site-packages\_pytest\config\__init__.py", line 166 in main
File "C:\actions-runner\_work\torch-xpu-ops\pytorch\third_party\torch-xpu-ops\test\xpu\run_test_win_with_skip_mtl.py", line 30 in

00007FF85B2E99C900007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FF85B1DE73E00007FF85B1DE700 python310.dll!PyFunction_Vectorcall [ @ ]
00007FF85B2E2A9100007FF85B2E2950 python310.dll!PyOS_URandomNonblock [ @ ]
00007FF85B2EB3A200007FF85B2EAFB0 python310.dll!PyEval_GetFuncDesc [ @ ]
00007FF85B2E77B700007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FF85B2E99C900007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FF85B1DE73E00007FF85B1DE700 python310.dll!PyFunction_Vectorcall [ @ ]
00007FF85B1E0A9100007FF85B1E06A0 python310.dll!PyCell_Set [ @ ]
00007FF85B1E0C4D00007FF85B1E0B90 python310.dll!PyMethod_Self [ @ ]
00007FF85B2E2A9100007FF85B2E2950 python310.dll!PyOS_URandomNonblock [ @ ]
00007FF85B2EB3A200007FF85B2EAFB0 python310.dll!PyEval_GetFuncDesc [ @ ]
00007FF85B2E77FC00007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FF85B2E99C900007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FF85B1DE73E00007FF85B1DE700 python310.dll!PyFunction_Vectorcall [ @ ]
00007FF85B1E0A9100007FF85B1E06A0 python310.dll!PyCell_Set [ @ ]
00007FF85B1E0C4D00007FF85B1E0B90 python310.dll!PyMethod_Self [ @ ]
00007FF85B2E2A9100007FF85B2E2950 python310.dll!PyOS_URandomNonblock [ @ ]
00007FF85B2EB3A200007FF85B2EAFB0 python310.dll!PyEval_GetFuncDesc [ @ ]
00007FF85B2E77FC00007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FF85B2E99C900007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FF85B1DE73E00007FF85B1DE700 python310.dll!PyFunction_Vectorcall [ @ ]
00007FF85B1E0A9100007FF85B1E06A0 python310.dll!PyCell_Set [ @ ]
00007FF85B1E0C4D00007FF85B1E0B90 python310.dll!PyMethod_Self [ @ ]
00007FF85B1DE4A100007FF85B1DE3E0 python310.dll!PyVectorcall_Call [ @ ]
00007FF85B2EB5B400007FF85B2EAFB0 python310.dll!PyEval_GetFuncDesc [ @ ]
00007FF85B2E654C00007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FF85B2E99C900007FF85B2E4670 python310.dll!PyEval_EvalFrameDefault [ @ ]
00007FF85B1DE73E00007FF85B1DE700 python310.dll!PyFunction_Vectorcall [ @ ]

Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, torch._C, torch._C._dynamo.autograd_compiler, torch._C._dynamo.eval_frame, torch._C._dynamo.guards, torch._C._dynamo.utils, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, multidict._multidict, yarl._quoting_c, propcache._helpers_c, aiohttp._http_writer, aiohttp._http_parser, aiohttp._websocket.mask, aiohttp._websocket.reader_c, frozenlist._frozenlist, charset_normalizer.md, charset_normalizer.cd, requests.packages.charset_normalizer.md, requests.packages.chardet.md, requests.packages.charset_normalizer.cd, requests.packages.chardet.cd, thriftpy2.transport.cybase, thriftpy2.transport.buffered.cybuffered, thriftpy2.transport.framed.cyframed, thriftpy2.transport.memory.cymemory, thriftpy2.transport.sasl.cysasl, thriftpy2.protocol.cybin.cybin, scipy._lib._ccallback_c, yaml._yaml, numba.core.typeconv._typeconv, numba._helperlib, numba._dynfunc, numba._dispatcher, numba.core.runtime._nrt_python, numba.np.ufunc._internal, numba.experimental.jitclass._box, numba.mviewbuf, numba.types.itertools, psutil._psutil_windows, scipy._lib._uarray._uarray, scipy.special._ufuncs_cxx, scipy.special._ufuncs, scipy.special._specfun, scipy.special._comb, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg._cythonized_array_utils, scipy.linalg._flinalg, scipy.linalg._solve_toeplitz, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_lapack, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.sparse.linalg._isolve._iterative, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, scipy.special._ellip_harm_2, scipy.signal._sigtools, scipy.signal._max_len_seq_inner, scipy.signal._upfirdn_apply, scipy.signal._spline, scipy.optimize._minpack2, scipy.optimize._group_columns, scipy._lib.messagestream, scipy.optimize._trlib._trlib, numpy.linalg.lapack_lite, scipy.optimize._lbfgsb, _moduleTNC, scipy.optimize._moduleTNC, scipy.optimize._cobyla, scipy.optimize._slsqp, scipy.optimize._minpack, scipy.optimize._lsq.givens_elimination, scipy.optimize._zeros, scipy.optimize.__nnls, scipy.optimize._highs.cython.src._highs_wrapper, scipy.optimize._highs._highs_wrapper, scipy.optimize._highs.cython.src._highs_constants, scipy.optimize._highs._highs_constants, scipy.linalg._interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, scipy.spatial._ckdtree, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._distance_wrap, scipy.spatial._hausdorff, scipy.spatial.transform._rotation, scipy.optimize._direct, scipy.integrate._odepack, scipy.integrate._quadpack, scipy.integrate._vode, scipy.integrate._dop, scipy.integrate._lsoda, scipy.interpolate._fitpack, scipy.interpolate.dfitpack, scipy.interpolate._bspl, scipy.interpolate._ppoly, scipy.interpolate.interpnd, scipy.interpolate._rbfinterp_pythran, scipy.interpolate._rgi_cython, scipy.signal._sosfilt, scipy.ndimage._nd_image, _ni_label, scipy.ndimage._ni_label, scipy.signal._spectral, scipy.special.cython_special, scipy.stats._stats, scipy.stats.beta_ufunc, scipy.stats._boost.beta_ufunc, scipy.stats.binom_ufunc, scipy.stats._boost.binom_ufunc, scipy.stats.nbinom_ufunc, scipy.stats._boost.nbinom_ufunc, scipy.stats.hypergeom_ufunc, scipy.stats._boost.hypergeom_ufunc, scipy.stats.ncf_ufunc, scipy.stats._boost.ncf_ufunc, scipy.stats.ncx2_ufunc, scipy.stats._boost.ncx2_ufunc, scipy.stats.nct_ufunc, scipy.stats._boost.nct_ufunc, scipy.stats.skewnorm_ufunc, scipy.stats._boost.skewnorm_ufunc, scipy.stats.invgauss_ufunc, scipy.stats._boost.invgauss_ufunc, scipy.stats._biasedurn, scipy.stats._levy_stable.levyst, scipy.stats._stats_pythran, scipy.stats._statlib, scipy.stats._mvn, scipy.stats._sobol, scipy.stats._qmc_cy, scipy.stats._rcont.rcont, scipy.signal._peak_finding_utils, markupsafe._speedups (total: 164)
The system cannot find the file specified.`
```

### Versions

pytorch: latest main
pytorch commit: torch-2.12.0a0+git6fe3e05