elastic / elastic/integrations
[Logstash] Always reports 0 for in memory queue
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
### Integration Name
Logstash [logstash]
### Dataset Name
logstash.node
### Integration Version
2.7.1
### Agent Version
8.18.
### Agent Output Type
elasticsearch
### Elasticsearch Version
8.18
### OS Version and Architecture
docker
### Software/API Version
_No response_
### Error Message
The field `logstash.node.stats.pipelines.queue.events_count` is always 0 for in memory queues, it is only populated when you use persistent disk. In general all `queue` metrics are empty for in memory queue:
```
"queue": {
"events_count": 0,
"max_queue_size_in_bytes": 0,
"queue_size_in_bytes": 0,
"type": "memory"
},
```
Taken from https://www.elastic.co/docs/reference/integrations/logstash/#node-stats
### Event Original
```
{
"ephemeral_id": "7114cd7d-8d91-4afc-a986-32487c3edcbe",
"events": {
"duration_in_millis": 191,
"filtered": 91,
"in": 95,
"out": 91,
"queue_push_duration_in_millis": 4
},
"hash": "0542fa70daa36dc3e858ea099f125cc8c9e451ebbfe8ea8867e52f9764da0a35",
"id": "pipeline-with-memory-queue",
"queue": {
"events_count": 0,
"max_queue_size_in_bytes": 0,
"queue_size_in_bytes": 0,
"type": "memory"
},
"reloads": {
"failures": 0,
"successes": 0
},
```
Directly form the documentation https://www.elastic.co/docs/reference/integrations/logstash/#node-stats
### What did you do?
Use the new logstash monitoring
### What did you see?
Missing `logstash.node.stats.pipelines.queue.events_count` apropriate value for that, representing the actual queue size.
### What did you expect to see?
The actual amount of docs in the in memory queue
### Anything else?
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.