influxdata / influxdata/influxdb

Replace scrapers functionality with tasks using prometheus.scrape

Open
#15,445 5 comments 0 reactions 0 assignees View on GitHub
area/2.x area/data ingestion area/tasks blocked-on/external
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

as a user
when i create and configure a scraper
then i would like a task created to actually scrape the data

so now that flux has the `prometheus.scrape` endpoint, we can replace the existing scraper functionality with Flux tasks.

Example Flux:
```
import "experimental/prometheus"

option task = {name: "Flux Scrape Data", every: 5s}

prometheus.scrape(url: "http://localhost:9999/metrics")
|> map(fn: (r) =>
({r with scrape_interval: string(v: task.every)}))
|> to(bucket: "flux_scrape_data", org: "influxdata")
```

and updated template dashboard:
[influxdb_2.0_oss_metrics_update.json.zip](https://github.com/influxdata/influxdb/files/3736835/influxdb_2.0_oss_metrics_update.json.zip)

Contributor guide

Open the contributing guide

Research direction

Start by locating the existing scraper functionality and how configured scrapers are executed. Compare that flow with the provided Flux prometheus.scrape task example, then review the linked dashboard template. Done means configured scrapers create tasks that scrape data through prometheus.scrape and the updated dashboard reflects the new task-based flow.

Written by the indexing model from the issue text.

Assessment

Tech stack
prometheus
Domain
backend, observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.