explosion / explosion/cython-blis

Baffling random failures in core Python behaviour in Windows CI

Open
#145 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
239
Forks
45
PR merge metrics
No merged PRs in 30d

Description

I'm seeing very frequent (once every 2-3 runs) random failures in CI, which happen
- exclusively on `windows-latest` runners, and
- exclusively on Python 3.13 and 3.14 (not 3.14t).
- I cannot yet say conclusively if they only happen in `actions/setup-python` or also inside `actions/cibuildwheel`.

The failures are as follows:

```python
C:\hostedtoolcache\windows\Python\3.13.9\x64\Lib\site-packages\hypothesis\statistics.py:90: in describe_statistics
runtime_ms = format_ms(t["runtime"] for t in cases)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

times = . at 0x00000200EA90CAD0>

def format_ms(times: Iterable[float]) -> str:
"""Format `times` into a string representing approximate milliseconds.

`times` is a collection of durations in seconds.
"""
ordered = sorted(times)
n = len(ordered) - 1
if n < 0 or any(math.isnan(t) for t in ordered): # pragma: no cover
return "NaN ms"
> lower = int(ordered[math.floor(n * 0.05)] * 1000)
^^^^^^^^^^^^^^^^^^^^
E OverflowError: cannot convert float infinity to integer

C:\hostedtoolcache\windows\Python\3.13.9\x64\Lib\site-packages\hypothesis\statistics.py:58: OverflowError
============================ slowest 10 durations =============================
1.06s call tests/test_gemm.py::test_threads_share_input
0.59s call tests/test_dotv.py::test_threads_share_input
0.42s call tests/test_dotv.py::test_memoryview_noconj
0.25s call tests/test_gemm.py::test_memoryview_notrans
```

`times` is a collection of runtimes for a test and I expect it to be fairly short - anywhere between 0 and a few thousands elements.
Note the underline, which is saying that `n * 0.05` is valued inf.

Either:
- `list.__len__` returned float inf instead of the expected (fairly small) integer, or
- a small integer * 0.05 returned inf.

Both of the above are absurd and I have no explanation.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Windows CI matrix and the failing tests tests/test_gemm.py::test_threads_share_input and tests/test_dotv.py::test_threads_share_input. Inspect Hypothesis statistics.py around describe_statistics and format_ms, then compare runs through actions/setup-python and actions/cibuildwheel. Done means identifying which path produces the infinite runtime value and documenting or addressing the reproducible cause.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
ci-cd, operating-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.