[Flaky Test] filebeat.tests.system.test_shutdown.Test test_shutdown_wait_ok_1_slab
- Dominant language
- Go
- Stars
- 12.7k
- Forks
- 5k
- Avg merge
- 2d 15m
- Merged PRs (30d)
- 385
Description
## Flaky Test
* **Test Name:** test_shutdown_wait_ok_1_slab
* **Scope:** filebeat.tests.system.test_shutdown.Test
* **File:** filebeat/tests/system/test_shutdown.py
* **Location:** filebeat/tests/system/test_shutdown.py:51
* **Buildkite Link:** https://buildkite.com/organizations/elastic/analytics/suites/beats/tests/d7b8b94b-2219-85ee-b9d4-544e362634da
* **Flaky Instances:** 1
* **Latest Occurrence:** 2026-06-25T04:33:03.790Z
### Details
```json
{
"id": "d7b8b94b-2219-85ee-b9d4-544e362634da",
"web_url": "https://buildkite.com/organizations/elastic/analytics/suites/beats/tests/d7b8b94b-2219-85ee-b9d4-544e362634da",
"scope": "filebeat.tests.system.test_shutdown.Test",
"name": "test_shutdown_wait_ok_1_slab",
"location": "filebeat/tests/system/test_shutdown.py:51",
"file_name": "filebeat/tests/system/test_shutdown.py",
"instances": 1,
"latest_occurrence_at": "2026-06-25T04:33:03.790Z",
"most_recent_instance_at": "2026-06-25T04:33:03.790Z",
"last_resolved_at": null,
"ownership_team_ids": [],
"failure_examples_count": 1
}
```
### Failure Examples
**Example 1:**
**Run:** https://api.buildkite.com/v2/analytics/organizations/elastic/suites/beats/runs/cc5703c9-217e-83ee-a80f-24f5f48bf27f
**Time:** 2026-06-25T04:06:48.823Z
**Stacktrace:**
```
a = (,)
@wraps(func)
def standalone_func(*a):
> return func(*(a + p.args), **p.kwargs)
../build/ve/linux/lib/python3.9/site-packages/parameterized/parameterized.py:518:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = , queue_type = 'slab'
@parameterized.expand(QUEUE_TYPES)
def test_shutdown_wait_ok(self, queue_type):
"""
Test stopping filebeat under load: wait for all events being published.
Runs under both queue implementations (memqueue and slabqueue).
"""
self.nasa_logs()
self.render_config_template(
path=os.path.abspath(self.working_dir) + "/log/*",
ignore_older="1h",
shutdown_timeout="10s",
rotate_every_kb=10000,
queue_type=queue_type,
)
filebeat = self.start_beat()
# Wait until first flush
self.wait_until(
lambda: self.log_contains_count("Publish event") > 200,
max_timeout=15)
filebeat.check_kill_and_wait()
self.get_log()
self.wait_log_contains(
"Output shutdown started.",
max_timeout=15)
self.wait_log_contains(
"Continue shutdown: All enqueued events have been published.",
max_timeout=15)
# validate registry entry offset matches last published event
registry = self.get_registry()
outputs = self.read_output()
offset = registry[0]["offset"]
assert len(registry) == 1
# we allow for a potential race in the harvester shutdown here.
# In some cases the registry offset might match the penultimate offset.
eol_offset = 1
if os.name == "nt":
eol_offset += 1
> assert (offset == (outputs[-1]["log.offset"] + eol_offset + len(outputs[-1]["message"])) or
offset == (outputs[-2]["log.offset"] + eol_offset + len(outputs[-2]["message"])))
E assert (219405 == ((219503 + 1) + 97) or 219405 == ((219405 + 1) + 97))
E + where 97 = len('204.212.132.27 - - [01/Jul/1995:00:34:04 -0400] "GET /images/KSC-logosmall.gif HTTP/1.0" 200 1204')
E + and 97 = len('204.212.132.27 - - [01/Jul/1995:00:34:04 -0400] "GET /images/NASA-logosmall.gif HTTP/1.0" 200 786')
tests/system/test_shutdown.py:99: AssertionError
```
Contributor guide
Research direction
Start with filebeat/tests/system/test_shutdown.py at the test_shutdown_wait_ok method and line 51, then reproduce the slab queue failure using the linked Buildkite run. Trace the registry and output offsets around shutdown and verify the test passes reliably with the expected final or penultimate event offset.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 62/100