graphile / graphile/worker

Add JS method to get the outstanding job count + queue depth

Open
#380 16 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
2.4k
Forks
126
Avg merge
2d 2h
Merged PRs (30d)
9

Description

### Feature description

I would like to be able to tell how many jobs are ready to be run in the `jobs` table. There is no public interface for doing that at this point.

### Motivating example

Our background workers run on "serverless" architecture with a minimal number of workers. We are occasionally faced with bursty background task (thousands or tens of thousands or jobs added) and would like to tell the underlying infrastructure that it needs to spin up additional workers temporarily to handle the load.

A public interface for querying the current count of outstanding jobs (ready to run + not locked or past their expiry time) would enable us to report graphile_worker metrics (useful for monitoring purposes anyway) and to spin up additional (temporary) workers (e.g., via having them `runOnce`).

### Alternatives considered

The existing `WorkerEvents` lets us know that there are no work items to be run but gives no insight into the job / queue depth. We could instrument a running job count ourselves using some separate table, but many of our jobs are fired off from postgres triggers which makes instrumentation a little more awkward and less explicit / clear to a new developer.

The new public `jobs` view is helpful and we can use that... I just thought a JS helper function might allow your users to navigate some of the more complex SQL gotchas and get counts that are more likely to be correct.

### Supporting development

I [tick all that apply]:

- [x] am interested in building this feature myself
- [x] am interested in collaborating on building this feature
- [x] am willing to help testing this feature before it's released
- [x] am willing to write a test-driven test suite for this feature (before it exists)
- [ ] am a [Graphile sponsor](https://www.graphile.org/sponsor/) ❤️
- [ ] have an active [support or consultancy contract](https://www.graphile.org/support/) with Graphile

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.