modelcontextprotocol / modelcontextprotocol/registry

Provision Grafana alert rules via Pulumi

Open
#1,214 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
7.3k
Forks
994
Avg merge
4d 12h
Merged PRs (30d)
19

Description

Context

Today only the Grafana datasources are provisioned in code (deploy/pkg/k8s/monitoring.go:546). Alert rules and dashboards live in Grafana's PostgreSQL DB and are edited via UI — not versioned, not reviewable, and lost if Grafana's DB ever resets.

Surfaced while triaging recurring Publish Endpoint Latency and Availability dropped below 95% alerts that turned out to be metric-pipeline artifacts post-deploy. Wanted to add for: 10m and noDataState: OK to both rules; can only do that via UI today.

Proposal

Add a grafana-alerts ConfigMap mounted at /etc/grafana/provisioning/alerting/ mirroring the existing datasources pattern, and move alert rules into it. Likely worth doing dashboards and notification policies at the same time.

Caveats

  • Provisioned alerts are read-only in the UI — every change goes through a PR thereafter. Worth confirming team is OK with that.
  • Existing rules need to be exported (Grafana provisioning API) and committed.

Acceptance

  • Alert rules and notification policies provisioned from code
  • Dashboards same (or explicitly out of scope)
  • deploy/README.md updated with the new edit workflow

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read deploy/pkg/k8s/monitoring.go around line 546 to follow the existing datasources provisioning pattern, then review the Grafana provisioning API for exporting current alert rules and notification policies. Update deploy/README.md with the edit workflow; done means the stated alerting resources are provisioned from code and dashboard scope is resolved.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql
Domain
devops, infrastructure, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.