Abhishek-Mallick / Abhishek-Mallick/universal-box

[TEMPLATE] - Node.js + Bull.js + Redis | Job Queues

Open
#161 0 comments 0 reactions 0 assignees View on GitHub
hacktoberfest
Dominant language
JavaScript
Stars
47
Forks
41
PR merge metrics
No merged PRs in 30d

Description

### What domain will the project template be based on?

Others (please specify in the details)

### If 'Others', please specify the preferred domain.

Job Queues

### Particular directory path for the template.

`templates/Job-Queues/Node-Bull-Redis/`

### What tech stacks do you want to include?

Node.js, Bull.js, Redis, Express.js, Bull Board or Arena

### Brief Description

This template demonstrates how to implement background processing in a Node.js application using Bull.js and Redis. It includes examples of creating job queues, processing jobs, and monitoring job statuses. This setup is ideal for handling tasks like sending emails, processing images, or any CPU-intensive operations without blocking the main thread.

**Key Features:**

1. Queue Configuration: Set up Bull queues connected to Redis for job management.
2. Job Producers: Implement API endpoints or services that add jobs to the queue.
3. Job Consumers: Create worker processes that consume and process jobs.
4. Monitoring Dashboard: Integrate Bull Board or Arena for real-time monitoring of the queues.
5. Error Handling: Implement retries, error logging, and failover mechanisms.
6. Graceful Shutdown: Ensure the application can shut down gracefully without losing jobs.

**Contributor Tasks:**

- Set up the Express.js server with endpoints for adding jobs.
- Implement sample jobs (e.g., email sending, data processing) with detailed comments.
- Configure the worker processes and ensure they handle concurrency properly.
- Integrate a monitoring tool and include instructions on accessing it.
- Write tests for job creation and processing using a testing framework like Jest.
- Update the README with setup instructions, including Redis installation and configuration.
- Create a corresponding markdown for populating [docs](https://universal-box.dev/) in `website/content/Templates/Job-Queues/Node-Bull-Redis.md` .. refer to [this one](https://github.com/Abhishek-Mallick/universal-box/blob/main/website/content/Templates/FullStack/EJS(SSR)%2BNodejs(Backend).md) for getting an idea on the format to follow

### 👀 Have you spent some time checking if this issue has been raised before?

- [X] Yes
- [ ] No

### 🏢 Have you read the Code of Conduct?

- [X] I have read the [Code of Conduct](https://github.com/Abhishek-Mallick/universal-box/blob/main/.github/CODE_OF_CONDUCT.md)

### Would you like to work on this issue?

No

Contributor guide

Open the contributing guide

Research direction

The template should be created in the `templates/Job-Queues/Node-Bull-Redis/` directory. Start by setting up an Express.js server with job endpoints, implementing sample jobs (like email sending), and configuring Bull.js workers with Redis. Integrate Bull Board or Arena for monitoring. Write tests with Jest. Finally, create a detailed README and a corresponding markdown file in `website/content/Templates/Job-Queues/Node-Bull-Redis.md` following the format of the linked example.

Written by the indexing model from the issue text.

Assessment

Tech stack
express, javascript, node.js, redis
Domain
backend, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.