how do you guys scale Kue ?

Hi Everyone, We have been using Kue for over 6 months now and with our growing needs, we need to distribute the workers over to multiple machines all connecting to a central redis.

We have been trying all the options but none seem to work so i guess it will be nice to understand how are you guys distributing the load of Kue workers to multiple machines.

Till now the biggest challenge is that when a worker from another machine connects to a central redis and whenever there is some problem with the connection it makes the whole kue unstable and the whole processing just stops.

We have tried reconnecting redis connections but it does not seem to work.
We also tried to use twemproxy, but that too i guess does not work with Kue.

I can see issues open with Redis connection but we are unable to find a simple way to horizontally scale Kue, any help here will be much appreciated.

The issue names no files, tests, or entry points. Begin by examining Kue's worker and Redis connection handling for multiple machines, then compare the behavior with the reported reconnecting and twemproxy attempts; completion requires an agreed scope and reproducible success criteria.

Written by the indexing model from the issue text.