[audit][tests-docs] Widespread sleep-based synchronization in autests (109 'sleep N' steps, 33 time.sleep calls) invites timing flakiness

Open
#59 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Active
Tech stack
python, shell
Domain
testing

Research direction

Start with tests/gold_tests/pluginTest/traffic_dump/traffic_dump.test.py around lines 343-362, then review the other named .test.py files and Python helpers for the reported sleep counts. Check the existing autest Ready, Condition-based file or log waiting, and traffic_ctl polling patterns first. Done means replacing timing-based synchronization with event or deadline-based waits and adding CI lint coverage for new bare sleep commands.

Written by the indexing model from the issue text.

Description

area:tests-docs audit severity:medium

Severity: medium · Category: testing
Location: tests/gold_tests/pluginTest/traffic_dump/traffic_dump.test.py:359

What's wrong

gold_tests contains 109 'sleep N' shell-command test steps in .test.py files plus 33 time.sleep() calls in Python helpers, many of which gate assertions on wall-clock guesses rather than events. traffic_dump.test.py is representative: after 'traffic_ctl plugin msg traffic_dump.limit 0' it runs a test step whose Command is literally 'sleep 2' (line 343, comment 'Give ATS some time to process the change') and later 'sleep 2' at line 359 followed immediately by 'file.Exists = False' for the expected-absent dump file — a negative assertion that passes vacuously if the plugin is merely slow, and six such sleep-2 steps exist in this one file. Other examples: block_errors.test.py uses six 'sleep 30' watcher processes; slice_prefetch.test.py:138 prefixes a curl with 'sleep 5'; stek_share.test.py:293 'sleep 10 && curl'. One test has already been permanently disabled for exactly this failure mode (see log_retention finding).

Evidence
traffic_dump.test.py:358-362: '# Sleep 2 seconds to give the replay plugin plenty of time to write the file.'
'tr.Processes.Default.Command = "sleep 2"' / 'file = tr.Disk.File(replay_file_session_12)' / 'file.Exists = False'.
// Counts: grep for sleep in *.test.py = 109 lines; time.sleep in tests/gold_tests/**/*.py = 33.
Suggested fix

Replace timing sleeps with condition polling: autest Ready conditions, Condition-based file/log waiting (ATS already has 'await' log-watch patterns), or traffic_ctl polling loops with deadline. Add an autest lint (tests/CI) flagging bare 'sleep' commands in new tests.


Filed from an automated multi-lens codebase audit. Full report: CODEBASE_AUDIT.md / audit-report.html on branch claude/codebase-audit-review-9nw7vz.

Dominant language
C++
Stars
0
Forks
0
Avg merge
8h 2m
Merged PRs (30d)
21

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from JakeChampion/trafficserver

All issues in JakeChampion/trafficserver

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.