prometheus / prometheus/client_python
Multiproc doesn't capture all metrics
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.4k
- Forks
- 876
- Avg merge
- 8d 4h
- Merged PRs (30d)
- 1
Description
Hey folks,
I have a similar setup to what is described here. The only difference is that I have nginx in front talking to gunicorn over sockets, but I don't think that matters as far as I can tell.
I wasn't able to use this code:
# Using multiprocess collector for registry
def make_metrics_app():
registry = CollectorRegistry()
multiprocess.MultiProcessCollector(registry)
return make_asgi_app(registry=registry)
metrics_app = make_metrics_app()
app.mount("/metrics", metrics_app)
It results in a page that gets a 307, so I implemented this:
@router.get("/metrics", response_class=PlainTextResponse, include_in_schema=False)
async def get_metrics() -> str:
"""
Get prometheus metrics
"""
registry = CollectorRegistry()
multiprocess.MultiProcessCollector(registry)
data = generate_latest(registry)
res = Response(content=data)
res.headers["Content-Type"] = CONTENT_TYPE_LATEST
res.headers["Content-Length"] = str(len(data))
return res
Apart from that my setup is pretty typical, I have some counters and a histogram:
AUTOMATIONS_TOTAL: Final = Gauge(
name=prefix_("automations_total"),
documentation="Number of automations run",
labelnames=["platform", "action", "kind", "status"],
multiprocess_mode="sum",
)
REQUEST_TIME_SECONDS: Final = Histogram(
name=prefix_("request_time_seconds"),
documentation="Time spent processing request",
labelnames=["method", "url_rule", "status_code"],
)
REQUESTS_IN_PROGRESS_TOTAL: Final = Gauge(
name=prefix_("requests_in_progress_total"),
documentation="Number of concurrent requests",
# See Metrics Tuning (Gauge)
# https://github.com/prometheus/client_python#multiprocess-mode-gunicorn
multiprocess_mode="sum",
)
My in progress counter always remains at zero, but I'm hoping that's because requests finish faster than I can see them. The histogram, however, never shows up - I see no record of it in my PROMETHEUS_MULTIPROC_DIR and it's never rendered in the output. Any ideas for what I could troubleshoot?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the FastAPI multiprocess collector path and the custom /metrics handler shown in the issue. Inspect CollectorRegistry, MultiProcessCollector, and the PROMETHEUS_MULTIPROC_DIR output while comparing the counter, gauge, and histogram behavior. Done means identifying why the histogram is absent and confirming the expected metrics are rendered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- fastapi, nginx, prometheus, python
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100