elastic / elastic/beats

[Meta][Metricbeat] - Collect even more additional Elasticsearch node metrics for enhanced dashboards

Open
#45,013 1 comment 0 reactions 0 assignees View on GitHub
Feature:Stack Monitoring Stalled Team:Monitoring
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

In order to improve stack monitoring, there needs to be some additional datapoints made available in a flattened view to enable visualizations and alerting.

Some of this data exists in the cluster_stats, but it being nested makes it very difficult to utilize.

Data that we would like to have either in the node_stats metricset (or a new metricset) is largely available by calling the _nodes API. Period for this might make sense to be different from existing node_stats as these items would not change often.

From the _nodes API

node.roles - this being stored in arrays, causes a single node to appear as multiple rows in a table visualization (one for each role), making it difficult to get a concise, single-line overview of a node. could this be as a single list instead?
node.attributes.availability_zone
node.attributes.region
node.instance_configuration - cloud only, but useful if available
node.version (i think this would be useful to track status of rolling updates)

node.os.cpu.processors: The number of logical CPU cores. This provides the denominator for calculating CPU load percentage, allowing for more effective alerting thresholds (e.g., load_avg_1m / processors > 0.8).

Thread Pool Statistics: Exposing the size and queue size for key thread pools is vital for diagnosing bottlenecks.
node.thread_pool.force_merge.size
node.thread_pool.force_merge.queue_size
node.thread_pool.get.size
node.thread_pool.get.queue_size
node.thread_pool.search.size
node.thread_pool.search.queue_size
node.thread_pool.write.size
node.thread_pool.write.queue_size
node.thread_pool.management.size
node.thread_pool.management.queue_size
node.thread_pool.esql_worker.size
node.thread_pool.esql_worker.queue_size
node.thread_pool.refresh.size
node.thread_pool.refresh.queue_size
node.thread_pool.snapshot.size
node.thread_pool.snapshot.queue_size

Quality-of-Life Improvement: Pre-calculated Percentages
As an additional, related enhancement, providing pre-calculated disk and memory usage percentages would be highly beneficial. While the raw used_in_bytes and total_in_bytes fields are available, exporting a calculated disk.used.pct and memory.used.pct field would simplify dashboard creation and alerting for all users.

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the Metricbeat node_stats metricset and existing cluster_stats data, then compare the available fields with the Elasticsearch _nodes API. Define which node roles, attributes, version, processor, thread-pool, disk, and memory values belong in the metricset, and verify that the resulting fields are flattened and suitable for dashboards and alerting.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.