elastic / elastic/elastic-agent

[Flaky Test] github.com/elastic/elastic-agent/testing/integration/ess TestSystemMetricsWithLogstashOutput

Open
#16,331 0 comments 0 reactions 0 assignees View on GitHub
flaky-test
Dominant language
Go
Stars
275
Forks
264
Avg merge
1d 20h
Merged PRs (30d)
303

Description

## Flaky Test

* **Test Name:** TestSystemMetricsWithLogstashOutput
* **Scope:** github.com/elastic/elastic-agent/testing/integration/ess
* **File:** N/A
* **Location:** N/A
* **Buildkite Link:** https://buildkite.com/organizations/elastic/analytics/suites/elastic-agent-ci/tests/c33300ef-c80d-8bbc-b600-841934df6f30
* **Flaky Instances:** 2
* **Latest Occurrence:** 2026-08-23T17:54:39.433Z

### Details

```json
{
"id": "c33300ef-c80d-8bbc-b600-841934df6f30",
"web_url": "https://buildkite.com/organizations/elastic/analytics/suites/elastic-agent-ci/tests/c33300ef-c80d-8bbc-b600-841934df6f30",
"scope": "github.com/elastic/elastic-agent/testing/integration/ess",
"name": "TestSystemMetricsWithLogstashOutput",
"location": null,
"file_name": null,
"instances": 2,
"latest_occurrence_at": "2026-08-23T17:54:39.433Z",
"most_recent_instance_at": "2026-08-23T17:54:39.433Z",
"last_resolved_at": null,
"ownership_team_ids": [],
"failure_examples_count": 1
}
```

### Failure Examples

**Example 1:**
**Run:** https://api.buildkite.com/v2/analytics/organizations/elastic/suites/elastic-agent-ci/runs/99b57209-9bdd-8451-a1d3-4f8ba8349d0b
**Time:** 2026-08-23T15:26:50.759Z
**Stacktrace:**

```
=== RUN TestSystemMetricsWithLogstashOutput
otel_test.go:3623:
Error Trace: /opt/buildkite-agent/builds/bk-agent-prod-gcp-1787499625944683626/elastic/elastic-agent-extended-testing/testing/integration/ess/otel_test.go:3623
Error: Received unexpected error:
docker compose up --detach --wait: exit status 1
Image busybox:latest Pulling
Image docker.elastic.co/logstash/logstash:9.2.2 Pulling
b05093807bb0 Pulling fs layer 0B
b05093807bb0 Downloading 32.12kB
b05093807bb0 Download complete 0B
b05093807bb0 Extracting 32.77kB
b05093807bb0 Extracting 2.226MB
b05093807bb0 Extracting 2.226MB
b05093807bb0 Pull complete 0B
Image busybox:latest Pulled
4888d4ca7933 Pulling fs layer 0B
4ca545ee6d5d Pulling fs layer 0B
0e8d997c145d Pulling fs layer 0B
e7f049d14054 Pulling fs layer 0B
7bc111d78429 Pulling fs layer 0B
f6794c675879 Pulling fs layer 0B
aa4aa2d5b430 Pulling fs layer 0B
49c3b461c78d Pulling fs layer 0B
a9ba155714c6 Pulling fs layer 0B
e7f049d14054 Waiting 0B
7bc111d78429 Waiting 0B
f6794c675879 Waiting 0B
a9ba155714c6 Waiting 0B
aa4aa2d5b430 Waiting 0B
49c3b461c78d Waiting 0B
4ca545ee6d5d Downloading 42B
4ca545ee6d5d Verifying Checksum 0B
4ca545ee6d5d Download complete 0B
0e8d997c145d Downloading 57.55kB
4888d4ca7933 Downloading 409.6kB
0e8d997c145d Verifying Checksum 0B
0e8d997c145d Download complete 0B
4888d4ca7933 Downloading 11.49MB
e7f049d14054 Downloading 540.7kB
4888d4ca7933 Downloading 27.87MB
e7f049d14054 Downloading 11.32MB
4888d4ca7933 Verifying Checksum 0B
4888d4ca7933 Download complete 0B
4888d4ca7933 Extracting 426kB
7bc111d78429 Downloading 21.6kB
e7f049d14054 Downloading 24.79MB
7bc111d78429 Verifying Checksum 0B
7bc111d78429 Download complete 0B
4888d4ca7933 Extracting 1.704MB
e7f049d14054 Downloading 40.96MB
e7f049d14054 Downloading 51.74MB
f6794c675879 Downloading 1.742kB
f6794c675879 Verifying Checksum 0B
f6794c675879 Download complete 0B
4888d4ca7933 Extracting 7.242MB
e7f049d14054 Downloading 62.54MB
aa4aa2d5b430 Downloading 287B
aa4aa2d5b430 Verifying Checksum 0B
aa4aa2d5b430 Download complete 0B
4888d4ca7933 Extracting 13.63MB
e7f049d14054 Downloading 74.37MB
4888d4ca7933 Extracting 20.45MB
e7f049d14054 Downloading 90.51MB
49c3b461c78d Downloading 287B
49c3b461c78d Verifying Checksum 0B
49c3b461c78d Download complete 0B
4888d4ca7933 Extracting 26.84MB
e7f049d14054 Downloading 105.1MB
a9ba155714c6 Downloading 730B
a9ba155714c6 Download complete 0B
4888d4ca7933 Extracting 32.8MB
e7f049d14054 Downloading 115.9MB
4888d4ca7933 Extracting 36.63MB
e7f049d14054 Downloading 123.5MB
4888d4ca7933 Extracting 38.34MB
e7f049d14054 Downloading 139.6MB
4888d4ca7933 Extracting 40.24MB
4888d4ca7933 Pull complete 0B
4ca545ee6d5d Extracting 42B
4ca545ee6d5d Extracting 42B
4ca545ee6d5d Pull complete 0B
0e8d997c145d Extracting 65.54kB
e7f049d14054 Downloading 155.2MB
0e8d997c145d Extracting 2.949MB
e7f049d14054 Downloading 170.7MB
0e8d997c145d Extracting 5.112MB
0e8d997c145d Extracting 5.387MB
0e8d997c145d Pull complete 0B
e7f049d14054 Downloading 179.4MB
e7f049d14054 Downloading 191.7MB
e7f049d14054 Downloading 206.3MB
e7f049d14054 Downloading 223MB
e7f049d14054 Downloading 238.1MB
e7f049d14054 Downloading 245.1MB
e7f049d14054 Downloading 257MB
e7f049d14054 Downloading 273.2MB
e7f049d14054 Downloading 287.8MB
e7f049d14054 Downloading 296.5MB
e7f049d14054 Downloading 306.2MB
e7f049d14054 Downloading 322.9MB
e7f049d14054 Downloading 339.6MB
e7f049d14054 Downloading 353.1MB
e7f049d14054 Downloading 362.8MB
e7f049d14054 Downloading 375.2MB
e7f049d14054 Downloading 389.2MB
e7f049d14054 Downloading 399.5MB
e7f049d14054 Downloading 410.9MB
e7f049d14054 Downloading 422.7MB
e7f049d14054 Downloading 438.4MB
e7f049d14054 Downloading 455.2MB
e7f049d14054 Verifying Checksum 0B
e7f049d14054 Download complete 0B
e7f049d14054 Extracting 557.1kB
e7f049d14054 Extracting 17.83MB
e7f049d14054 Extracting 34.54MB
e7f049d14054 Extracting 49.58MB
e7f049d14054 Extracting 66.29MB
e7f049d14054 Extracting 82.44MB
e7f049d14054 Extracting 90.24MB
e7f049d14054 Extracting 96.93MB
e7f049d14054 Extracting 102.5MB
e7f049d14054 Extracting 108.6MB
e7f049d14054 Extracting 115.3MB
e7f049d14054 Extracting 122MB
e7f049d14054 Extracting 128.1MB
e7f049d14054 Extracting 134.3MB
e7f049d14054 Extracting 139.8MB
e7f049d14054 Extracting 145.4MB
e7f049d14054 Extracting 150.4MB
e7f049d14054 Extracting 155.4MB
e7f049d14054 Extracting 163.2MB
e7f049d14054 Extracting 177.7MB
e7f049d14054 Extracting 194.4MB
e7f049d14054 Extracting 211.1MB
e7f049d14054 Extracting 220MB
e7f049d14054 Extracting 235.6MB
e7f049d14054 Extracting 236.2MB
e7f049d14054 Extracting 236.7MB
e7f049d14054 Extracting 237.9MB
e7f049d14054 Extracting 238.4MB
e7f049d14054 Extracting 239MB
e7f049d14054 Extracting 239.5MB
e7f049d14054 Extracting 240.1MB
e7f049d14054 Extracting 252.3MB
e7f049d14054 Extracting 255.1MB
e7f049d14054 Extracting 271.8MB
e7f049d14054 Extracting 288.6MB
e7f049d14054 Extracting 305.8MB
e7f049d14054 Extracting 316.4MB
e7f049d14054 Extracting 327MB
e7f049d14054 Extracting 338.1MB
e7f049d14054 Extracting 344.8MB
e7f049d14054 Extracting 351.5MB
e7f049d14054 Extracting 354.3MB
e7f049d14054 Extracting 364.9MB
e7f049d14054 Extracting 369.9MB
e7f049d14054 Extracting 376MB
e7f049d14054 Extracting 389.4MB
e7f049d14054 Extracting 401.6MB
e7f049d14054 Extracting 403.3MB
e7f049d14054 Extracting 405.5MB
e7f049d14054 Extracting 408.3MB
e7f049d14054 Extracting 411.1MB
e7f049d14054 Extracting 416.1MB
e7f049d14054 Extracting 422.2MB
e7f049d14054 Extracting 423.4MB
e7f049d14054 Extracting 423.9MB
e7f049d14054 Extracting 425MB
e7f049d14054 Extracting 425.6MB
e7f049d14054 Extracting 440.1MB
e7f049d14054 Extracting 442.3MB
e7f049d14054 Extracting 442.9MB
e7f049d14054 Extracting 445.1MB
e7f049d14054 Extracting 456.2MB
e7f049d14054 Extracting 456.8MB
e7f049d14054 Extracting 457MB
e7f049d14054 Pull complete 0B
7bc111d78429 Extracting 32.77kB
7bc111d78429 Extracting 2.119MB
7bc111d78429 Extracting 2.119MB
7bc111d78429 Pull complete 0B
f6794c675879 Extracting 1.742kB
f6794c675879 Extracting 1.742kB
f6794c675879 Pull complete 0B
aa4aa2d5b430 Extracting 287B
aa4aa2d5b430 Extracting 287B
aa4aa2d5b430 Pull complete 0B
49c3b461c78d Extracting 287B
49c3b461c78d Extracting 287B
49c3b461c78d Pull complete 0B
a9ba155714c6 Extracting 730B
a9ba155714c6 Extracting 730B
a9ba155714c6 Pull complete 0B
Image docker.elastic.co/logstash/logstash:9.2.2 Pulled
Network elastic-agent-compose-4166466260_default Creating
Network elastic-agent-compose-4166466260_default Creating
Network elastic-agent-compose-4166466260_default Created
Network elastic-agent-compose-4166466260_default Created
Container elastic-agent-compose-4166466260-init-logstash-1 Creating
Container elastic-agent-compose-4166466260-init-logstash-1 Created
Container elastic-agent-compose-4166466260-logstash-1 Creating
Container elastic-agent-compose-4166466260-logstash-1 Created
Container elastic-agent-compose-4166466260-init-logstash-1 Starting
Container elastic-agent-compose-4166466260-init-logstash-1 Started
Container elastic-agent-compose-4166466260-init-logstash-1 Waiting
Container elastic-agent-compose-4166466260-init-logstash-1 Exited
Container elastic-agent-compose-4166466260-logstash-1 Starting
Container elastic-agent-compose-4166466260-logstash-1 Started
Container elastic-agent-compose-4166466260-logstash-1 Waiting
Container elastic-agent-compose-4166466260-init-logstash-1 Waiting
Container elastic-agent-compose-4166466260-init-logstash-1 Exited
container elastic-agent-compose-4166466260-logstash-1 exited (1)
Test: TestSystemMetricsWithLogstashOutput
--- FAIL: TestSystemMetricsWithLogstashOutput (27.80s)

```

Contributor guide

Open the contributing guide

Research direction

Start with TestSystemMetricsWithLogstashOutput in testing/integration/ess/otel_test.go, especially the failure at line 3623, and review the linked Buildkite failure. Run the test and inspect the docker compose setup; done means the test no longer flakes during environment startup and passes reliably.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker-compose, go
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.