Intermittent metric registration error: worker_millis_per_event
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 88
Description
**Problem**:
Intermittent warning during pipeline startup. The worker_millis_per_event metric fails to register but pipeline functions normally.
```[org.logstash.execution.AbstractPipelineExt] Metric registration error: `worker_millis_per_event` could not be registered in namespace `[:stats, :pipelines, :logstash, :plugins, :filters, :c13c16377fd9a15e6e691284665f2da3494b42273b1fc6e37b6330fff12eb5ab, :flow]```
**Expected versus actual behavior**:
Expected: All flow metrics register successfully; /_node/stats/pipelines returns full pipeline statistics.
Actual: Registration fails. Pipelines start and process events normally (status: "green", Pipelines running {count: 2}). However, /_node/stats/pipelines returns empty pipeline data: {"pipelines": {}}.
**Environment Details**:
1. Logstash version - 9.0.3 OSS Version
2. Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker) - Built from source
3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes) - Kubernetes (running as a process with logstash binary in Docker container)
**JVM**:
bash-4.4$ java -version
openjdk version "21.0.12" 2026-07-21
OpenJDK Runtime Environment (build 21.0.12+8-suse-01500-x8664)
OpenJDK 64-Bit Server VM (build 21.0.12+8-suse-01500-x8664, mixed mode, sharing)
**OS version**:
`Linux log-transformer-597b585774 6.4.0-150600.23.42-default #1 SMP PREEMPT_DYNAMIC Fri Mar 7 09:53:00 UTC 2025 (7bf6ecd) x86_64 x86_64 x86_64 GNU/Linux`
**Steps to reproduce**:
Not consistently reproducible - Occurs intermittently during startup. The reported error is observed approximately once in every 40-50 startups. Reproduced with automated script that deploys logstash, searches for error pattern during startup, redeploys if error not found. This continues till error pattern is found.
**Additional Observations**:
1. No functional impact - Pipeline processes events normally
2. Metrics not available - Pipeline stats are empty from endpoint localhost:9600/_node/stats/pipelines
{
"host": "log-transformer",
"version": "9.0.3",
"http_address": "0.0.0.0:9600",
"id": "5fbd8fxxxxxxxxxxxx5602",
"name": "log-transformer",
"ephemeral_id": "36axxxxxxxxxxxxxxxxx2348a7",
"snapshot": false,
"status": "unknown",
"pipeline": {
"workers": 2,
"batch_size": 2048,
"batch_delay": 50
},
"pipelines": {}
}
3. Self-resolving: Logstash process restart with same configuration succeeds without error. No warning message and stats are also available in /_node/stats/pipelines endpoint.
Note - We have created a discussion in elastic community (https://discuss.elastic.co/t/intermittent-metric-registration-error-worker-millis-per-event/383836) and tried the workaround/suggestions provided there. But unfortunately, it didn't solve our issue.
**Logs**:
Full startup logs attached - [Logstash Logs - Metrics registration error · GitHub](https://gist.github.com/Mariyabanatic/70f79f24f3cb10eff92bbc24829ce12e)
Contributor guide
Research direction
Start with the attached startup logs and the automated redeploy loop described in the issue, then inspect the metric-registration path for worker_millis_per_event and the /_node/stats/pipelines endpoint. Done means the intermittent registration warning is eliminated and the endpoint consistently returns populated pipeline statistics across repeated startups.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100