elastic / elastic/logstash

Intermittent metric registration error: worker_millis_per_event

Open
#19,491 0 comments 0 reactions 0 assignees View on GitHub
bug status:needs-triage
Dominant language
Java
Stars
14.9k
Forks
3.5k
Avg merge
1d 4h
Merged PRs (30d)
88

Description

**Problem**:

Intermittent warning during pipeline startup. The worker_millis_per_event metric fails to register but pipeline functions normally.

```[org.logstash.execution.AbstractPipelineExt] Metric registration error: `worker_millis_per_event` could not be registered in namespace `[:stats, :pipelines, :logstash, :plugins, :filters, :c13c16377fd9a15e6e691284665f2da3494b42273b1fc6e37b6330fff12eb5ab, :flow]```

**Expected versus actual behavior**:

Expected: All flow metrics register successfully; /_node/stats/pipelines returns full pipeline statistics.

Actual: Registration fails. Pipelines start and process events normally (status: "green", Pipelines running {count: 2}). However, /_node/stats/pipelines returns empty pipeline data: {"pipelines": {}}.

**Environment Details**:

1. Logstash version - 9.0.3 OSS Version
2. Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker) - Built from source
3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes) - Kubernetes (running as a process with logstash binary in Docker container)

**JVM**:

bash-4.4$ java -version
openjdk version "21.0.12" 2026-07-21
OpenJDK Runtime Environment (build 21.0.12+8-suse-01500-x8664)
OpenJDK 64-Bit Server VM (build 21.0.12+8-suse-01500-x8664, mixed mode, sharing)

**OS version**:

`Linux log-transformer-597b585774 6.4.0-150600.23.42-default #1 SMP PREEMPT_DYNAMIC Fri Mar 7 09:53:00 UTC 2025 (7bf6ecd) x86_64 x86_64 x86_64 GNU/Linux`

**Steps to reproduce**:

Not consistently reproducible - Occurs intermittently during startup. The reported error is observed approximately once in every 40-50 startups. Reproduced with automated script that deploys logstash, searches for error pattern during startup, redeploys if error not found. This continues till error pattern is found.

**Additional Observations**:

1. No functional impact - Pipeline processes events normally
2. Metrics not available - Pipeline stats are empty from endpoint localhost:9600/_node/stats/pipelines
{
"host": "log-transformer",
"version": "9.0.3",
"http_address": "0.0.0.0:9600",
"id": "5fbd8fxxxxxxxxxxxx5602",
"name": "log-transformer",
"ephemeral_id": "36axxxxxxxxxxxxxxxxx2348a7",
"snapshot": false,
"status": "unknown",
"pipeline": {
"workers": 2,
"batch_size": 2048,
"batch_delay": 50
},
"pipelines": {}
}

3. Self-resolving: Logstash process restart with same configuration succeeds without error. No warning message and stats are also available in /_node/stats/pipelines endpoint.

Note - We have created a discussion in elastic community (https://discuss.elastic.co/t/intermittent-metric-registration-error-worker-millis-per-event/383836) and tried the workaround/suggestions provided there. But unfortunately, it didn't solve our issue.

**Logs**:

Full startup logs attached - [Logstash Logs - Metrics registration error · GitHub](https://gist.github.com/Mariyabanatic/70f79f24f3cb10eff92bbc24829ce12e)

Contributor guide

Open the contributing guide

Research direction

Start with the attached startup logs and the automated redeploy loop described in the issue, then inspect the metric-registration path for worker_millis_per_event and the /_node/stats/pipelines endpoint. Done means the intermittent registration warning is eliminated and the endpoint consistently returns populated pipeline statistics across repeated startups.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.