jd / jd/tenacity

Three repr tests assert nothing; wait_incrementing's max cap and the idle_for statistic have no test that pins them

Open
#715 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8.8k
Forks
359
Avg merge
1m
Merged PRs (30d)
1

Description

While running a mutation-based check of the test suite at `3e58094` a few gaps came up that seemed worth a note. None of this is a bug in tenacity — every "break" below was deliberate, and the shipped code is correct. The core is well guarded: flipping the `stop_after_attempt` boundary reddened 28 tests, disabling `retry_if_exception_type` 36, `reraise=True` 8, the `attempt_number` statistic 12.

### 1. `repr` tests that cannot fail

`TestBase::test_retrying_repr`, `test_asyncio.py::TestAsyncio::test_repr` and `test_tornado.py::TestTornado::test_repr` each call `repr(...)` and assert nothing. With `BaseRetrying.__repr__` changed to return `""` all three still pass. They do catch a repr that raises, but not one that is wrong. `test_callstate_repr` two lines below the first one asserts on the string and is the shape these could take.

### 2. `wait_incrementing`'s `max` is never exercised

The only test (`test_incrementing_sleep`) passes no `max`. Removing the cap — `max(0, min(result, self.max))` → `max(0, result)` — survives the suite. With `max=3` the strategy should return `[1, 2, 3, 3, 3, 3]` across attempts 1–6; the mutant returns `[1, 2, 3, 4, 5, 6]`. One extra case with `max=` and an attempt past it would cover it.

### 3. `statistics["idle_for"]` is only ever asserted with `mock.ANY`

All four assertions on the statistics dict use `"idle_for": mock.ANY`, so never accumulating it (`self.statistics["idle_for"] += sleep` → `pass`) survives. In those tests the wait is fixed, so the expected total is known and could be pinned.

### 4. Two tests with no possible red

`test_legacy_explicit_stop_type` and `test_legacy_explicit_wait_type` construct `Retrying(stop="stop_after_attempt")` / `Retrying(wait="exponential_sleep")` (with a `type: ignore`) and assert nothing. There is no string-handling code in `BaseRetrying.__init__` to break, so the tests can only fail if the constructor starts rejecting strings. If legacy string support was removed at some point, these may be leftovers.

Full write-up with the mutation spec and per-test results, reproducible against `3e58094`: https://github.com/ArnauFerma/falsifiable-tests/blob/main/case-studies/tenacity.md

Happy to send a PR for 1–3 if you'd like one.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the three repr tests named in the issue, plus test_incrementing_sleep and the statistics assertions; compare them with test_callstate_repr and the existing wait and retry tests. Run the focused tests, then add assertions for repr output, the wait_incrementing max sequence, and the exact idle_for total. Review the two legacy string-constructor tests to determine whether they still have meaningful coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.