bcgov / bcgov/SIMS

Update process-notification - too many requests error

Open
#6,409 2 comments 0 reactions 1 assignee Claimed by @uhelm View on GitHub
Business Queue Consumers
Dominant language
TypeScript
Stars
29
Forks
15
Avg merge
1d 14h
Merged PRs (30d)
38

Description

**User Story**
GCNotify has an API rate limit of 1000 calls per minute, rolling timeframe.
Every minute the `process-notification` job creates a new job to send requests up to the polling limit.
Other jobs (like the SIN response) generates more than 1000 email notifications when it is finished.
When we're over 1000 emails, the second `process-notification` job starts and we very quickly get a 529 error for too many requests.
When these error, an email notification is sent to RequestIT via Sysdig. Currently, we're receiving this error every day.
Too many error notifications when _no error occurs_ can lead to extra work and "alert noise" which can delay actual errors from being seen.

**Acceptance Criteria**
- [ ] Update the process-notification to only allow one active job at a time.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.