[release/2.14] [Bug Skip]: torch-xpu-ops.test.xpu.test_cuda_multigpu_xpu.TestCudaMultiGPU.test_mem_get_info failed at assert error not less thank expected value
- Dominant language
- Python
- Stars
- 113
- Forks
- 128
- Avg merge
- 5d 9h
- Merged PRs (30d)
- 112
Description
### 🐛 Describe the bug with skip template
AssertionError: 68702699520 not less than 68702699520 To execute this test,
## Affected Tests
Cases:
~~op_ut,third_party.torch-xpu-ops.test.xpu.test_cuda_multigpu_xpu.TestCudaMultiGPU,test_mem_get_info~~
## Error Message
```
Traceback (most recent call last): File "/__w/torch-xpu-ops/torch-xpu-ops/pytorch/third_party/torch-xpu-ops/test/xpu/test_cuda_multigpu_xpu.py", line 1122, in test_mem_get_info _test(0) File "/__w/torch-xpu-ops/torch-xpu-ops/pytorch/third_party/torch-xpu-ops/test/xpu/test_cuda_multigpu_xpu.py", line 1118, in _test self.assertLess(after_free_bytes, before_free_bytes) File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/unittest/case.py", line 1232, in assertLess self.fail(self._formatMessage(msg, standardMsg)) File "/tmp/xpu-tool/Python/3.10.20/x64/lib/python3.10/unittest/case.py", line 675, in fail raise self.failureException(msg) AssertionError: 68702699520 not less than 68702699520 To execute this test, run the following from the base repo dir: PYTORCH_TEST_WITH_SLOW=1 python test/xpu/test_cuda_multigpu_xpu.py TestCudaMultiGPU.test_mem_get_info This message can be suppressed by setting PYTORCH_PRINT_REPRO_ON_FAILURE=0
```
## Test Code Snippet
```python
def test_mem_get_info(self):
def _test(device: str | int | torch.device):
# Prevent PyTorch from reusing the allocated memory
torch.accelerator.empty_cache()
torch.accelerator.synchronize()
before_free_bytes, before_available_bytes = torch.get_device_module(
GPU_TYPE
).mem_get_info(device)
# increasing to 8MB to force acquiring a new block and overcome blocksize differences across platforms
t = torch.randn(1024 * 1024 * 8, device=device) # noqa: F841
if IS_JETSON or TEST_XPU:
# w/o syncing, mem_get_info will run before memory allocated has actually increased.
# This race condition causes consistent failure
torch.accelerator.synchronize()
after_free_bytes, after_available_bytes = torch.get_device_module(
GPU_TYPE
).mem_get_info(device)
self.assertLess(after_free_bytes, before_free_bytes)
self.assertEqual(before_available_bytes, after_available_bytes)
# Test calls with different device representations
_test(0)
_test(torch.device(GPU_TYPE))
_test(torch.device(f"{GPU_TYPE}:0"))
_test(GPU_TYPE)
_test(f"{GPU_TYPE}:0")
if TEST_MULTIGPU:
_test(1)
_test(torch.device(f"{GPU_TYPE}:1"))
_test(f"{GPU_TYPE}:1")
```
### Test location
test\xpu\test_cuda_multigpu_xpu.py
```python
def test_external_streams(self):
device = torch.get_device_module(GPU_TYPE).device(0)
with self._get_external_stream(device) as stream_v:
```
test case failed at
```
self.assertLess(after_free_bytes, before_free_bytes)
```
# TODO XPU PR
https://github.com/intel/torch-xpu-ops/pull/4294
### Versions
PyTorch: main 2.14.0a0
XPU Tests: torch-xpu-ops.test.xpu.test_cuda_multigpu_xpu.TestCudaMultiGPU.test_mem_get_info
Contributor guide
Research direction
Start with test/xpu/test_cuda_multigpu_xpu.py and run PYTORCH_TEST_WITH_SLOW=1 python test/xpu/test_cuda_multigpu_xpu.py TestCudaMultiGPU.test_mem_get_info. Read the _test helper around the failing assert and the mem_get_info behavior before and after allocation. Done means the test reliably observes reduced free memory while the available-memory assertion still passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100