Automattic / Automattic/kue

Dynamic queue

Open
#1,235 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
9.4k
Forks
858
PR merge metrics
No merged PRs in 30d

Description

Guys, its possible do something like that?

I want to have many queues, if i have a sequence of messages to send. Those messages have to be sent in the right sequence to the same number, but the sequence is not important between numbers.

This way works, but a think, it probably leaves a lot of leaks on the memory. Because of `queue.process(job.data.phone.toString(), 1, processMessage)` who may not be used after the conversation ends, for example

message-sender
```js
const arr = Array.from(new Array(5))
let phones = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
phones.forEach(async phone => {
arr.forEach(async (_, key) => {
let message = `${key}\t`
queue.create('managequeue', { phone, message })
.removeOnComplete(true)
.save( function(err){
if( err ) console.log(err);
})
})
})
```

woker
```js
const phones = []

queue.process('managequeue', 1, function(job, done){
queue.create(job.data.phone.toString(), { phone: job.data.phone, message: job.data.message })
.removeOnComplete(true)
.save()

if(!queues.includes(job.data.phone)) {
queue.process(job.data.phone.toString(), 1, processMessage)
phones.push(job.data.phone)
}

done()
})

```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the worker snippet's queue.process calls and the per-phone processor registration shown in the issue. Investigate the processor lifecycle after a conversation ends and whether the message-sender and worker snippets reproduce the reported memory concern. Done would require a clearly supported way to preserve ordering per phone without retaining unused processors.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js, redis
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.