NVIDIA / NVIDIA/cccl

[BUG]: Sporadic test failure in test_unique_by_key.py::test_unique_by_key

Open
#4,203 0 comments 1 reaction 1 assignee Claimed by @NaderAlAwar View on GitHub
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

### Is this a duplicate?

- [x] I confirmed there appear to be no [duplicate issues](https://github.com/NVIDIA/cccl/issues) for this bug and that I agree to the [Code of Conduct](CODE_OF_CONDUCT.md)

### Type of Bug

Runtime Error

### Component

cuda.parallel (Python)

### Describe the bug

Test failure observed during CI run:

```
> np.testing.assert_array_equal(h_out_keys, expected_keys)
tests/test_unique_by_key.py:149:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/tmp/cuda_parallel_venv/lib/python3.10/site-packages/numpy/_utils/__init__.py:85: in wrapper
return fun(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
args = (, array([ 9, 1, 14, ..., 19, 9, 14], dtype=uint16), array([ 9, 1, 14, ..., 19, 9, 14], dtype=uint16))
kwds = {'err_msg': '', 'header': 'Arrays are not equal', 'strict': False, 'verbose': True}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E AssertionError:
E Arrays are not equal
E
E Mismatched elements: 254 / 15938702 (0.00159%)
E Max absolute difference among violations: 247
E Max relative difference among violations: 1.
E ACTUAL: array([ 9, 1, 14, ..., 19, 9, 14], dtype=uint16)
E DESIRED: array([ 9, 1, 14, ..., 19, 9, 14], dtype=uint16)
/usr/lib/python3.10/contextlib.py:79: AssertionError
=========================== short test summary info ============================
FAILED tests/test_unique_by_key.py::test_unique_by_key[uint16-16777216] - AssertionError:
Arrays are not equal
Mismatched elements: 254 / 15938702 (0.00159%)
Max absolute difference among violations: 247
Max relative difference among violations: 1.
ACTUAL: array([ 9, 1, 14, ..., 19, 9, 14], dtype=uint16)
DESIRED: array([ 9, 1, 14, ..., 19, 9, 14], dtype=uint16)
============= 1 failed, 706 passed, 2 xfailed in 614.65s (0:10:14) =============
```

This test failure is not easily reproducible locally. This issue is file to document the failure and to track future diagnostic efforts.

### How to Reproduce

Assuming randomness is in out favor, the failure may be reproduced as follows:

```sh
cd cccl/python/cuda_parallel
pytest tests/test_unique_by_key.py::test_unique_by_key
```

### Expected behavior

Test should not fail

### Reproduction link

_No response_

### Operating System

Ubuntu 22.04

### nvidia-smi output

```
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 565.57.01 Driver Version: 565.57.01 CUDA Version: 12.7 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 2080 Off | 00000000:0A:00.0 Off | N/A |
| 41% 33C P8 2W / 215W | 2MiB / 8192MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+
```

### NVCC version

NVCC from CUDAToolkit 12.8 was used.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.