elastic / elastic/integrations
[ER] Add module remote_read capabilities to Promethues integration
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 225
Description
**Describe the enhancement:**
Today the Elastic stack is interacting with Prometheus in a “one way” mode only.
Through the Prometheus [module](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-prometheus.html) , metricbeat allow Elastic users to write Prometheus data into Elasticsearch.
Those data are available to be used in Kibana. To implement the writing of Prometheus data, Metricbeat implements half of the Prometheus storage [API](https://prometheus.io/docs/prometheus/latest/storage/#remote-storage-integrations), the “remote write” [part](https://prometheus.io/docs/concepts/remote_write_spec/). Because Metricbeat implements only half of the API, Prometheus server cannot “read back” the data stored in Elasticsearch. And thus do not qualify as a remote storage for Prometheus.
**Describe a specific use case for the enhancement or feature:**
We would like metricbeat to implement the remote read part of Prometheus [API](https://prometheus.io/blog/2019/10/10/remote-read-meets-streaming/ ). Then, Prometheus could use Elasticsearch as a remote store.
Samples of such implementation [exists](https://github.com/prometheus/prometheus/tree/main/documentation/examples/remote_storage/remote_storage_adapter/influxdb ).
Similar tor the “write” [endpoint]( https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-prometheus-remote_write.html ) metricbeat would expose a “read” endpoint :
```
- module: prometheus
metricsets: ["remote_read"]
host: "localhost"
port: "9201"
```
**Benefits:**
* Migration project from Prometheus to Elastic on huge and complex environment can be tough and providing a smooth way to migrate the backend without touching the ingestion workflow is a significant value add to complex project as there is no disruption in the overall query workflow process.
* It will also minimize the end-user change management.
* Present Elastic a full compatible backend for Prometheus
Contributor guide
Assessment
This issue has not been assessed yet.