influxdata / influxdata/telegraf
A new plugin inputs.opensearch with the RemoteStats API
- Dominant language
- Go
- Stars
- 17.8k
- Forks
- 5.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 161
Description
### Use Case
Hello Team,
Currently, the following inputs plugins are available for use with the OpenSearch service: `inputs.elasticsearch`, `inputs.elasticsearch_query`, `inputs.opensearch_query`. It is possible to use the elasticsearch plugins with the OpenSearch service with some caveats, as the APIs are aligned (new just need to replace appropriate strings).
While the OpenSearch service is being developed, a new API comes into play that we are also interested in scraping for the running nodes:
- [Segment Replication API](https://opensearch.org/docs/latest/api-reference/cat/cat-segment-replication/)
```
GET /_cat/segment_replication
GET /_cat/segment_replication/{index}
```
- [Remote Store Stats API](https://opensearch.org/docs/latest/tuning-your-cluster/availability-and-recovery/remote-store/remote-store-stats-api/)
```
GET _remotestore/stats/
GET _remotestore/stats//
```
I briefly checked Elasticsearch for the corresponding metrics API and it seems that there is no 1-to-1 match for the metrics we need. So I propose to add a new `opensearch` plugin to inputs, so we can query new metrics.
I have a draft of changes to demonstrate the possible solution of integrating new metrics to the elasticsearch plugin, please let me know if you are iterested in introducing a new inputs plugin and I need to submit a PR>
Configuration Example:
```
[[inputs.elasticsearch]]
tagexclude = []
servers = [ "http://127.0.0.1:9200",]
local = true
ccr_stats = true
cluster_health = true
cluster_health_level = "indices"
cluster_stats = true
[processors]
[[processors.strings]]
namepass = [ "elasticsearch_*",]
[[processors.strings.replace]]
measurement = "*"
old = "elasticsearch_"
new = "opensearch_"
```
### Expected behavior
A new `opensearch` plugin is added to handle OpenSearch API specifics.
### Actual behavior
Not able to scrape new opensearch metrics, need to replace `elasticsearch_` with `opensearch_`.
https://opensearch.org/docs/latest/api-reference/cat/cat-segment-replication/
https://opensearch.org/docs/latest/tuning-your-cluster/availability-and-recovery/remote-store/remote-store-stats-api/
### Additional info
_No response_
Contributor guide
Research direction
Review the existing inputs.elasticsearch, inputs.elasticsearch_query, and inputs.opensearch_query plugins, then read the linked Segment Replication and Remote Store Stats API documentation. Define the new plugin's scope around collecting the requested OpenSearch metrics and verify that its configuration and measurements work against the documented endpoints.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability-sre
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100