getodk / getodk/central

Expose more information about state of worker queue

Open
#1,240 0 comments 0 reactions 0 assignees View on GitHub
backend
Dominant language
JavaScript
Stars
226
Forks
235
Avg merge
2d 22h
Merged PRs (30d)
19

Description

I was watching the `pg_stat_activity` on the QA server when pm2 was running one instance. I observed the following (copying from Slack):

> Even though there was only one pm2 instance, I saw 5 different queries at about the same time from a worker checking for work. I would only expect there to be 1. We spin up a new worker if the existing worker seems to be stuck, but I’m realizing that we never spin down the new worker. So a server could have a growth in the number of workers over time. I’m not sure that’s really the explanation here though, because the 5 queries were all in the same second.

> The 5 queries stuck around in `pg_stat_activity` for a few seconds. I’d expect them to be nearly instantaneous. Their state was idle, so maybe that has something to do with it.

> Follow-up idea here: add an option for more verbose logging or other monitoring of the workers. It’d be nice to have more information about the number of workers and how long each of their queries is taking, including the check for work. Right now, it can be hard to gain insight into what’s happening in the worker queue.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.