Three repr tests assert nothing; wait_incrementing's max cap and the idle_for statistic have no test that pins them
- Dominant language
- Python
- Stars
- 8.8k
- Forks
- 359
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
While running a mutation-based check of the test suite at `3e58094` a few gaps came up that seemed worth a note. None of this is a bug in tenacity — every "break" below was deliberate, and the shipped code is correct. The core is well guarded: flipping the `stop_after_attempt` boundary reddened 28 tests, disabling `retry_if_exception_type` 36, `reraise=True` 8, the `attempt_number` statistic 12.
### 1. `repr` tests that cannot fail
`TestBase::test_retrying_repr`, `test_asyncio.py::TestAsyncio::test_repr` and `test_tornado.py::TestTornado::test_repr` each call `repr(...)` and assert nothing. With `BaseRetrying.__repr__` changed to return `""` all three still pass. They do catch a repr that raises, but not one that is wrong. `test_callstate_repr` two lines below the first one asserts on the string and is the shape these could take.
### 2. `wait_incrementing`'s `max` is never exercised
The only test (`test_incrementing_sleep`) passes no `max`. Removing the cap — `max(0, min(result, self.max))` → `max(0, result)` — survives the suite. With `max=3` the strategy should return `[1, 2, 3, 3, 3, 3]` across attempts 1–6; the mutant returns `[1, 2, 3, 4, 5, 6]`. One extra case with `max=` and an attempt past it would cover it.
### 3. `statistics["idle_for"]` is only ever asserted with `mock.ANY`
All four assertions on the statistics dict use `"idle_for": mock.ANY`, so never accumulating it (`self.statistics["idle_for"] += sleep` → `pass`) survives. In those tests the wait is fixed, so the expected total is known and could be pinned.
### 4. Two tests with no possible red
`test_legacy_explicit_stop_type` and `test_legacy_explicit_wait_type` construct `Retrying(stop="stop_after_attempt")` / `Retrying(wait="exponential_sleep")` (with a `type: ignore`) and assert nothing. There is no string-handling code in `BaseRetrying.__init__` to break, so the tests can only fail if the constructor starts rejecting strings. If legacy string support was removed at some point, these may be leftovers.
Full write-up with the mutation spec and per-test results, reproducible against `3e58094`: https://github.com/ArnauFerma/falsifiable-tests/blob/main/case-studies/tenacity.md
Happy to send a PR for 1–3 if you'd like one.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the three repr tests named in the issue, plus test_incrementing_sleep and the statistics assertions; compare them with test_callstate_repr and the existing wait and retry tests. Run the focused tests, then add assertions for repr output, the wait_incrementing max sequence, and the exact idle_for total. Review the two legacy string-constructor tests to determine whether they still have meaningful coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 74/100