document recommendations for reverse proxying prometheus
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 717
- Forks
- 1.2k
- Avg merge
- 3d 15h
- Merged PRs (30d)
- 19
Description
It would be good if there were some recommendations documented about which settings to use when putting prometheus behind a reverse proxy like nginx.
I'm thinking of stuff like this:
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $hostname;
(Haven't checked that this is a sensible assortment for prometheus specifically.
But then there is also the /api/v1/notifications/live endpoint which uses Server-sent events which requires special care so that is passes through nginx.
https://prometheus.io/docs/prometheus/latest/querying/api/#live-notifications
If this isn't configured correctly there will be a notification as such after a short while:
Real-time notifications interrupted.
With the following config it hasn't popped up yet. But i haven't checked how sensible every single one of those settings are:
upstream prometheus {
server [::1]:9090 ;
}
server {
listen 0.0.0.0:80 ;
listen [::0]:80 ;
server_name prometheus.example.org ;
location / {
return 301 https://$host$request_uri;
}
}
server {
listen 0.0.0.0:443 ssl ;
listen [::0]:443 ssl ;
server_name prometheus.example.org ;
http2 on;
ssl_certificate /var/lib/acme/prometheus.example.org/fullchain.pem;
ssl_certificate_key /var/lib/acme/prometheus.example.org/key.pem;
ssl_trusted_certificate /var/lib/acme/prometheus.example.org/chain.pem;
location / {
proxy_pass http://prometheus;proxy_set_header
# this is the code snippet in the top of the issue:
include /nix/store/bjs5fy079yy18vqcfhzqw4011kc2rw04-nginx-recommended-proxy_set_header-headers.conf;
}
location =/api/v1/notifications/live {
proxy_pass http://prometheus;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_buffering off;
proxy_cache off;
proxy_read_timeout 10m;
proxy_send_timeout 10m;
gzip off;
# this is the code snippet in the top of the issue:
include /nix/store/bjs5fy079yy18vqcfhzqw4011kc2rw04-nginx-recommended-proxy_set_header-headers.conf;
}
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Prometheus HTTP API live notifications page and the reverse-proxy configuration shown in the issue. Determine which nginx settings are required for normal requests and for SSE at /api/v1/notifications/live. Done means adding documentation with reliable reverse-proxy recommendations, including SSE handling that avoids the interruption warning.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- nginx, prometheus
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100