Not enough PostgreSQL connections

I am posting here first to ask if other people even have this issue - since the last few updates, our GitLab has increasingly produced errors randomly, and we finally narrowed it down to exhausting the PostgreSQL max_connection default setting of 100. This is a tricky problem, because a git clone might fail, or a random page on your server may render wrong or show an error, but a reload/retry will usually fix it, so it's horrible to debug. You can check if this is happening to you with a command like:

docker exec -it gitlab-server grep "sorry, too many clients already" /home/git/gitlab/log/application_json.log

We have about 15-20 users (some inactive), so I would expect this to be happening to many people by now. If you run an install around that size, please report here if you are seeing any errors like that.

Assuming that multiple people report back that they are seeing this in their logs, I suggest raising max_connections to 150 or 200, which can be done in multiple ways, but since the Postgres-Repository is abandoned, the simplest way for this image would be to add:

    command:
    - '-c max_connections=200'

The required connections don't seem to directly scale with users; gitlab.com itself only uses 500 according to some discussions in issues, my guess is that it's more closely related to used functionality where each aspect of GitLab (such as issues, pipelines) in use is responsible for a few open connections. There were mentions in patch notes the past where raising the limit was suggested.

Start by checking the Docker image's PostgreSQL configuration and the application_json.log entry for “sorry, too many clients already.” Verify whether the reported connection exhaustion is reproducible and determine the appropriate configuration change. Done means the change is documented or implemented and GitLab no longer intermittently fails under the described workload.