How to processing multiple jobs in a batch?
- Dominant language
- JavaScript
- Stars
- 9.4k
- Forks
- 858
- PR merge metrics
- No merged PRs in 30d
Description
Assuming we have many jobs of insert data to a database in the queue, instead of process each jobs one by one, which is not very efficient, I expect to be able to grab multiple jobs, and insert those data in one go by using db's bulk insert command. How can I do that?
Is there anyway to fetch all jobs(or a given number of jobs) from queue, process them and then mark them as done? Thanks.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the existing job-consumption API and Redis-backed worker behavior. Define how a worker fetches a bounded batch, processes the jobs with one bulk database operation, handles failures, and marks completion. Done means the behavior is documented and covered for successful and partially failing batches.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js, redis
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100