elastic / elastic/beats

[Metricbeat] Add Prometheus module remote_read capabilities

Open
#35,762 7 comments 3 reactions 0 assignees View on GitHub
needs_team
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

**Describe the enhancement:**
Today the Elastic stack is interacting with Prometheus in a “one way” mode only.
Through the Prometheus [module](https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-prometheus.html) , metricbeat allow Elastic users to write Prometheus data into Elasticsearch.
Those data are available to be used in Kibana. To implement the writing of Prometheus data, Metricbeat implements half of the Prometheus storage [API](https://prometheus.io/docs/prometheus/latest/storage/#remote-storage-integrations), the “remote write” [part](https://prometheus.io/docs/concepts/remote_write_spec/). Because Metricbeat implements only half of the API, Prometheus server cannot “read back” the data stored in Elasticsearch. And thus do not qualify as a remote storage for Prometheus.

**Describe a specific use case for the enhancement or feature:**
We would like metricbeat to implement the remote read part of Prometheus [API](https://prometheus.io/blog/2019/10/10/remote-read-meets-streaming/ ). Then, Prometheus could use Elasticsearch as a remote store.
Samples of such implementation [exists](https://github.com/prometheus/prometheus/tree/main/documentation/examples/remote_storage/remote_storage_adapter/influxdb ).
Similar tor the “write” [endpoint]( https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-prometheus-remote_write.html ) metricbeat would expose a “read” endpoint :
```
- module: prometheus
metricsets: ["remote_read"]
host: "localhost"
port: "9201"
```
**Benefits:**
* Migration project from Prometheus to Elastic on huge and complex environment can be tough and providing a smooth way to migrate the backend without touching the ingestion workflow is a significant value add to complex project as there is no disruption in the overall query workflow process.
* It will also minimize the end-user change management.
* Present Elastic a full compatible backend for Prometheus

Contributor guide

Open the contributing guide

Research direction

Start by reading the Prometheus remote storage API documentation and the existing Metricbeat Prometheus remote_write module and endpoint. Define the remote_read metricset using the requested host and port configuration, then verify that Prometheus can query data stored in Elasticsearch without changing its ingestion workflow.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, prometheus
Domain
backend-api-design, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.