[meta] Flakey bundled plugins from #logstash-plugin-ci

# Flaky test analysis — logstash-plugins scheduled CI failures

Source: `oblt-robot-ci` Slack failure notifications. Investigation via `gh` CLI (read-only)
against each run's `--log-failed` output and job matrix. Date of analysis: 2026-08-20.

## Method

For each run the failed-job log was downloaded and the RSpec summary, failing example
name, and error signature extracted. The job matrix was inspected to see whether the
failure was isolated to a single stack/branch cell (a strong flaky signal) or reproduced
across all cells.

## Classification key

- **Test race** — intermittent, timing-dependent assertion or socket/thread race in the spec.
- **Concurrency** — thread-safety defect (unsynchronized shared state) crashing a worker thread.
- **Hang/timeout** — job killed by the 60-minute step timeout; test deadlocks/hangs.
- **Infra** — CI environment failure (Docker image pull i/o timeout, missing DRA build, setup step); not a plugin/test bug.

## Summary

| Plugin | Runs (IDs) | Failing test / step | Signature | Verdict |Assignee|PR|Merged|
|---|---|---|---|---|---|---|---|
| logstash-output-tcp | 31783808676, 32009100050 | `spec/outputs/tcp_spec.rb:194` register | `register` raises `ConfigurationError`; `ECONNREFUSED` on client connect | Flaky — test race |@andsel|https://github.com/logstash-plugins/logstash-output-tcp/pull/73| ✅ |
| logstash-integration-snmp | 31783846781, 32009105537, 32347392451 | Setup / install step | `Install failed with an unexpected code: 1` (Docker) | Flaky — infra |@andsel|https://github.com/logstash-plugins/logstash-integration-snmp/pull/98| ✅|
| logstash-input-elasticsearch | 31784088299 | Setup / image pull | `docker.elastic.co … i/o timeout` (ES 9.5.1) | Flaky — infra |@andsel| NO FIX | [spurious](https://github.com/elastic/logstash/issues/19452#issuecomment-5476527008)|
| logstash-output-elasticsearch | 31784049577, 31873619940, 32347652832 | Integration specs (index template / bulk) | `Connection refused … port 9200` — ES not ready | Flaky — infra/env |@andsel| https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/1285 | ✅|
| logstash-input-tcp | 31784048596, 32114939230, 32231316548 | Run tests step | `timed out after 60 minutes`; connect-refused retry loop | Flaky — hang/timeout |@andsel|[issue](https://github.com/logstash-plugins/logstash-input-tcp/issues/285) [PR](https://github.com/logstash-plugins/logstash-input-tcp/pull/286)|✅|
| logstash-input-ganglia | 31873726335, 31935726847, 32115237611 | `spec/inputs/ganglia_spec.rb:61/73` | UDP "should receive the correct data/value/name" | Flaky — test race ||||
| logstash-filter-translate | 31873735909, 31935728649, 32115232167 | `spec/filters/scheduling_spec.rb:197` | reload "expected 11, got 1" | Flaky — test race |@andsel|https://github.com/logstash-plugins/logstash-filter-translate/pull/117| ✅|
| logstash-mixin-scheduler | 31873773968, 31935751921, 32009785840 | Run tests step | `timed out after 60 minutes` | Flaky — hang/timeout |@andsel | [issue](https://github.com/logstash-plugins/logstash-mixin-scheduler/issues/9) ||
| logstash-output-http | 31935473675 | `spec/outputs/http_spec.rb:496` | "keeps retrying due a protocol exception" | Flaky — test race |@andsel| [investigation PR](https://github.com/logstash-plugins/logstash-output-http/pull/163) [issue](https://github.com/logstash-plugins/logstash-output-http/issues/164) https://github.com/logstash-plugins/logstash-output-http/pull/165 ||
| logstash-input-file | 31935589000, 32114901771 | `spec/inputs/file_read_spec.rb:345` | "cleans up sincedb entry" | Flaky — test race |@andsel| [issue](https://github.com/logstash-plugins/logstash-input-file/issues/337) [PR](https://github.com/logstash-plugins/logstash-input-file/pull/338) ||
| logstash-input-elastic_serverless_forwarder | 31935741498 | SSL client-auth cert-rejection examples | intermittent `bad_certificate`/`certificate_required` | Flaky — test race (TLS) ||||
| logstash-output-lumberjack | 32009744611 | Run tests step | `0 failures` but exit 1; server thread `RuntimeError: crashed` | Flaky — thread exception leak | @donoghuc | https://github.com/logstash-plugins/logstash-output-lumberjack/pull/37 | ✅|
| logstash-integration-kafka | 32009180665, 32409673361 | `spec/integration/outputs/kafka_spec.rb:177` | "distribute events to all partitions"; `Unable to poll Kafka consumer` + Fatal assertion | Flaky — test race ||||
| logstash-input-beats | 32009491400 | Run tests step | `0 failures` but exit 1; `Zlib::GzipFile::Error: Unexpected end of ZLIB input stream` | Flaky — test race |@andsel|https://github.com/logstash-plugins/logstash-input-beats/pull/578|✅ |
| logstash-input-redis | 32114916361 | `spec/inputs/redis_spec.rb:35` | "should read events from a list"; `events[1000]` nil → `NoMethodError` | Flaky — test race ||||
| logstash-input-http_poller | 32115257090 | Setup test environment | single cell failed in 9s, 15 cells passed | Flaky — infra/setup |||
| logstash-codec-multiline | 32231288604 | `spec/codecs/identity_map_codec_spec.rb` | `ConcurrencyError: … unsynchronized modifications` | Flaky — concurrency |@andsel|https://github.com/logstash-plugins/logstash-codec-multiline/pull/79|✅ |
| logstash-input-http | 32347870269 | Setup / image pull | `docker.elastic.co … i/o timeout` (logstash 8.19.20) | Flaky — infra ||||

All 18 plugins are flaky; none show a deterministic, reproducible product bug. They split
into three buckets: **test races/concurrency** (11), **hang/timeout** (2), and **CI infra**
(5).

---

## Per-plugin detail

### logstash-output-tcp — flaky (test race)
- Runs: [31783808676](https://github.com/logstash-plugins/logstash-output-tcp/actions/runs/31783808676) (Stack 8.current), [32009100050](https://github.com/logstash-plugins/logstash-output-tcp/actions/runs/32009100050) (Stack 9.previous snapshot).
- Only one matrix cell failed; every other stack/branch passed the same commit.
- `36 examples, 1 failure`. Failing example at `spec/outputs/tcp_spec.rb:194`:
`expect { subject.register }.to_not raise_error` → `got LogStash::ConfigurationError`.
- Also `Errno::ECONNREFUSED` connecting to `localhost:1000` from `tcp_spec.rb:54`.
- Signature is a register/SSL-setup + client-connect race in the spec harness, not a code defect.

### logstash-integration-snmp — flaky (infra)
- Runs: [31783846781](https://github.com/logstash-plugins/logstash-integration-snmp/actions/runs/31783846781) (unit), [32009105537](https://github.com/logstash-plugins/logstash-integration-snmp/actions/runs/32009105537) (integration), [32347392451](https://github.com/logstash-plugins/logstash-integration-snmp/actions/runs/32347392451) (integration).
- No RSpec failures. Fails at environment setup with `Install failed with an unexpected code: 1`.
- Docker/DRA image acquisition failure — CI infrastructure, not a test bug.

### logstash-input-elasticsearch — flaky (infra)
- Run: [31784088299](https://github.com/logstash-plugins/logstash-input-elasticsearch/actions/runs/31784088299).
- `failed to do request: Head "https://docker.elastic.co/v2/elasticsearch/elasticsearch/manifests/9.5.1": dial tcp … i/o timeout`.
- Image-pull timeout during setup. Infra.

### logstash-output-elasticsearch — flaky (infra/env)
- Runs: [31784049577](https://github.com/logstash-plugins/logstash-output-elasticsearch/actions/runs/31784049577), [31873619940](https://github.com/logstash-plugins/logstash-output-elasticsearch/actions/runs/31873619940), [32347652832](https://github.com/logstash-plugins/logstash-output-elasticsearch/actions/runs/32347652832) (all integration).
- `136 examples, 1–2 failures`. Repeated `curl: (7) Failed to connect to elasticsearch port 9200: Connection refused`.
- The ES container is not ready/healthy when the integration specs run — environment timing, not plugin logic.

### logstash-input-tcp — flaky (hang/timeout)
- Runs: [31784048596](https://github.com/logstash-plugins/logstash-input-tcp/actions/runs/31784048596) (main snapshot), [32114939230](https://github.com/logstash-plugins/logstash-input-tcp/actions/runs/32114939230) (9.previous), [32231316548](https://github.com/logstash-plugins/logstash-input-tcp/actions/runs/32231316548) (9.previous).
- `The action 'Run tests' has timed out after 60 minutes.`
- Preceded by `Connection refused - connect(2) for "127.0.0.1" port …` retry loops; a spec deadlocks/hangs on a socket, never completing.

### logstash-input-ganglia — flaky (test race)
- Runs: [31873726335](https://github.com/logstash-plugins/logstash-input-ganglia/actions/runs/31873726335), [31935726847](https://github.com/logstash-plugins/logstash-input-ganglia/actions/runs/31935726847), [32115237611](https://github.com/logstash-plugins/logstash-input-ganglia/actions/runs/32115237611).
- `7 examples, 1 failure`, and the failing example varies between cells: `ganglia_spec.rb:61` "should receive the correct data", `:73` "should receive the value", and "should receive the name".
- UDP receive-timing race: the assertion runs before the datagram is processed. The varying failing example across cells confirms non-determinism.

### logstash-filter-translate — flaky (test race)
- Runs: [31873735909](https://github.com/logstash-plugins/logstash-filter-translate/actions/runs/31873735909), [31935728649](https://github.com/logstash-plugins/logstash-filter-translate/actions/runs/31935728649), [32115232167](https://github.com/logstash-plugins/logstash-filter-translate/actions/runs/32115232167).
- `55 examples, 1 failure` at `spec/filters/scheduling_spec.rb:197` — "scheduled reloading merge uses the merged dictionary after reload":
`expected: "11", got: "1"`.
- The assertion samples the dictionary before the scheduled reload thread has merged the update — scheduler timing race.

### logstash-mixin-scheduler — flaky (hang/timeout)
- Runs: [31873773968](https://github.com/logstash-plugins/logstash-mixin-scheduler/actions/runs/31873773968), [31935751921](https://github.com/logstash-plugins/logstash-mixin-scheduler/actions/runs/31935751921), [32009785840](https://github.com/logstash-plugins/logstash-mixin-scheduler/actions/runs/32009785840).
- `The action 'Run tests' has timed out after 60 minutes.` A scheduler spec hangs and never returns.

### logstash-output-http — flaky (test race)
- Run: [31935473675](https://github.com/logstash-plugins/logstash-output-http/actions/runs/31935473675).
- `264 examples, 1 failure` at `spec/outputs/http_spec.rb:496` — "with supported_protocols set to (disabled) 1.1 keeps retrying due a protocol exception".
- Retry-count/timing expectation that depends on how many retries elapse in a fixed window — race.

### logstash-input-file — flaky (test race)
- Runs: [31935589000](https://github.com/logstash-plugins/logstash-input-file/actions/runs/31935589000), [32114901771](https://github.com/logstash-plugins/logstash-input-file/actions/runs/32114901771).
- `143 examples, 1 failure` at `spec/inputs/file_read_spec.rb:345` — "delete on complete with sincedb cleanup enabled cleans up sincedb entry".
- Filesystem/sincedb-flush timing race between the reader thread and the assertion.

### logstash-input-elastic_serverless_forwarder — flaky (test race, TLS)
- Run: [31935741498](https://github.com/logstash-plugins/logstash-input-elastic_serverless_forwarder/actions/runs/31935741498).
- `36 examples, 1 failure, 2 pending`. Failing examples rotate among the SSL client-authentication cert-rejection cases (`ssl_client_authentication => required/optional`, self-signed / mismatched SAN, `ssl_verification_mode => full`) expecting a `bad_certificate`/`certificate_required` rejection.
- TLS handshake-timing race: the rejection sometimes surfaces as a different/absent error than asserted.

### logstash-output-lumberjack — flaky (thread exception leak)
- Run: [32009744611](https://github.com/logstash-plugins/logstash-output-lumberjack/actions/runs/32009744611).
- RSpec reports `2 examples, 0 failures` but the job exits `1`.
- Lumberjack server threads terminate with `RuntimeError: crashed` / `Whoa we shouldn't get this frame:` (`jls-lumberjack … server.rb:70`). These are deliberately-injected failures in `lumberjack_spec.rb`, but with `report_on_exception` the thread crash escapes and forces a non-zero process exit — an unclean-teardown flake, not an assertion failure.

### logstash-integration-kafka — flaky (test race)
- Runs: [32009180665](https://github.com/logstash-plugins/logstash-integration-kafka/actions/runs/32009180665) (main), [32409673361](https://github.com/logstash-plugins/logstash-integration-kafka/actions/runs/32409673361) (11.x).
- `44 examples, 1 failure` at `spec/integration/outputs/kafka_spec.rb:177` — "when using multi partition topic should distribute events to all partitions".
- `Unable to poll Kafka consumer` and `Exception in thread "kafka-input-worker-test-0" java.lang.AssertionError: Fatal assertion` — broker/partition-assignment timing race; partition distribution not yet balanced when asserted.

### logstash-input-beats — flaky (test race)
- Run: [32009491400](https://github.com/logstash-plugins/logstash-input-beats/actions/runs/32009491400).
- Both cells report `262 examples, 0 failures`, yet the job is red.
- `Zlib::GzipFile::Error: Unexpected end of ZLIB input stream` — a compressed-payload spec crashed on a truncated/incomplete stream (network/timing), aborting the process after the summary.

### logstash-input-redis — flaky (test race)
- Run: [32114916361](https://github.com/logstash-plugins/logstash-input-redis/actions/runs/32114916361).
- `6 examples, 1 failure` at `spec/inputs/redis_spec.rb:35` — "should read events from a list":
`expect(events[1000].get('sequence')).to eq(1000)` → `NoMethodError` (`events[1000]` is nil).
- Not all 1001 events were delivered before the assertion; also `IOError: closed stream` during teardown. Data-delivery timing race.

### logstash-input-http_poller — flaky (infra/setup)
- Run: [32115257090](https://github.com/logstash-plugins/logstash-input-http_poller/actions/runs/32115257090).
- Only `Stack 9.current` (5.x branch) failed, at `Setup test environment` after 9s; the other 15 matrix cells passed. Environment-setup infra flake.

### logstash-codec-multiline — flaky (concurrency)
- Run: [32231288604](https://github.com/logstash-plugins/logstash-codec-multiline/actions/runs/32231288604).
- Exit code 1 with `ConcurrencyError: Detected invalid array contents due to unsynchronized modifications with concurrent users` from the `IdentityMapCodec#map_cleanup` thread (`spec/codecs/identity_map_codec_spec.rb:134`).
- Genuine thread-safety defect in the identity-map cleanup path exercised by concurrent spec threads; manifests intermittently.

### logstash-input-http — flaky (infra)
- Run: [32347870269](https://github.com/logstash-plugins/logstash-input-http/actions/runs/32347870269).
- `docker.elastic.co/logstash/logstash:8.19.20 … Head … i/o timeout`. Image-pull timeout during setup. Infra.

---

## Notes

- The five **infra** cases (snmp, input-elasticsearch, output-elasticsearch, http_poller, input-http) are Docker image-pull / container-readiness / setup timeouts and are not fixable in plugin code; they recur across many repos and point at registry/runner flakiness.
- The **hang/timeout** cases (input-tcp, mixin-scheduler) waste a full 60-minute runner each; worth prioritising as they are the most expensive.
- `logstash-codec-multiline`'s `ConcurrencyError` is the one signature that reflects a real (thread-safety) bug rather than pure test timing, even though it surfaces intermittently.