internetarchive / internetarchive/openlibrary
Investigate updating `gunicorn` worker counts
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
### Proposal
@cdrini points out that many of our `gunicorn` workers are sitting idle waiting for requests. As per [`gunicorn`'s guidance](https://docs.gunicorn.org/en/latest/design.html#how-many-workers), we may be using too many workers.
Let's investigate lower the number of `gunicorn` workers in our servers.
### Justification
Having too many idle workers can negatively affect site performance.
Adding more [statsD instrumentation](https://docs.gunicorn.org/en/latest/instrumentation.html) could help us determine if we should adjust the worker count, and by how much.
### Breakdown
#### Requirements Checklist
* [ ]
#### Related files
*
#### Stakeholders
*
#### Instructions for Contributors
Please [run these commands](https://github.com/internetarchive/openlibrary/wiki/Git-Cheat-Sheet#working-on-your-branch) to ensure your repository is up to date **before** [creating a new branch](https://github.com/internetarchive/openlibrary/wiki/Git-Cheat-Sheet#making-changes-and-creating-a-pull-request) to work on this issue and **each time after** pushing code to Github, because the pre-commit bot may add commits to your PRs upstream.
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by reviewing the current Gunicorn server configuration alongside Gunicorn's worker-count and statsD instrumentation guidance, then measure worker utilization and site performance. Done means a documented worker-count recommendation supported by measurements, with any needed instrumentation identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100