elastic / elastic/integrations

[ER] Add module remote_read capabilities to Promethues integration

Open
#6,562 5 comments 0 reactions 0 assignees View on GitHub
Integration:prometheus Stalled
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
2d 17h
Merged PRs (30d)
225

Description

**Describe the enhancement:**
Today the Elastic stack is interacting with Prometheus in a “one way” mode only.
Through the Prometheus [module](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-prometheus.html) , metricbeat allow Elastic users to write Prometheus data into Elasticsearch.
Those data are available to be used in Kibana. To implement the writing of Prometheus data, Metricbeat implements half of the Prometheus storage [API](https://prometheus.io/docs/prometheus/latest/storage/#remote-storage-integrations), the “remote write” [part](https://prometheus.io/docs/concepts/remote_write_spec/). Because Metricbeat implements only half of the API, Prometheus server cannot “read back” the data stored in Elasticsearch. And thus do not qualify as a remote storage for Prometheus.

**Describe a specific use case for the enhancement or feature:**
We would like metricbeat to implement the remote read part of Prometheus [API](https://prometheus.io/blog/2019/10/10/remote-read-meets-streaming/ ). Then, Prometheus could use Elasticsearch as a remote store.
Samples of such implementation [exists](https://github.com/prometheus/prometheus/tree/main/documentation/examples/remote_storage/remote_storage_adapter/influxdb ).
Similar tor the “write” [endpoint]( https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-prometheus-remote_write.html ) metricbeat would expose a “read” endpoint :
```
- module: prometheus
metricsets: ["remote_read"]
host: "localhost"
port: "9201"
```
**Benefits:**
* Migration project from Prometheus to Elastic on huge and complex environment can be tough and providing a smooth way to migrate the backend without touching the ingestion workflow is a significant value add to complex project as there is no disruption in the overall query workflow process.
* It will also minimize the end-user change management.
* Present Elastic a full compatible backend for Prometheus

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.