prometheus / prometheus/client_python
Add usage example for Celery
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.4k
- Forks
- 876
- Avg merge
- 8d 4h
- Merged PRs (30d)
- 1
Description
I recently wanted to collect custom application metrics from a rather large Celery app. I tried several approaches with varying degrees of awkwardness, until finally settling on having a "sidecar" HTTP server process for all Celery nodes.
This works pretty well, but took me a while to get right. To spare others from wasting as much time as me, I would like to propose adding a section on getting Celery metrics set up with the Prometheus client. I wrote up the solution on StackOverflow: https://stackoverflow.com/a/75799358/2532203
If you're interested, I can properly summarise it and open a PR?
Also, if you look at the code, I needed to copy some swaths of the library because the built-in Prometheus WSGI server is hard-coded to demonise the thread; this causes trouble when running in a demonised process. Maybe this could be made optional with another keyword argument that defaults to True?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the StackOverflow solution linked in the issue and the Prometheus client's exposition.py WSGI server reference. Summarize the sidecar HTTP-server setup for collecting metrics from Celery workers in the appropriate documentation section, and verify that the example is complete and understandable. Treat the optional daemon-thread behavior as a separate question unless the existing discussion resolves it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- prometheus, python
- Domain
- documentation, observability-sre
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100