Unitech / Unitech/pm2

PM2 puts every other request in queue even with lots of workers free

Open
#4,768 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stale
Dominant language
JavaScript
Stars
43.3k
Forks
2.7k
PR merge metrics
No merged PRs in 30d

Description

What's going wrong?

pm2 puts every other request in queue instead of calling another free worker. very strange behavior.

My api has only one route. Each request takes between 1-2 min to completely resolve and send back the response. As soon as I make my first request, I can see in the pm2 logs that the request has been accepted, but if I make a second request to the same route it gets queued and only gets processed after the first is completed. Only in case I make a third request to the same route while the second request is in queue, another worker is called and accepts the third request, the second stays in queue until the first gets resolved. I hope I made myself clear

first request is accepted promptly by a worker, second request gets queued and third request gets also promply accepted by another worker, fourth gets queued, fifth gets accepted, sixth gets queued and so on.

I have 24 available workers.

How could we reproduce this issue?

here is my very simple server:

const express = require('express');
const runner = require('./Rrunner2');
const bodyParser = require('body-parser');



const app = express();
app.use(express.json({limit: '50mb', extended: true}));
app.use(bodyParser.json());


app.post('/optimize', (req, res) => {
    try{
        
        console.log(`Worker process ID - ${process.pid} has accepted the request.`);
    
        const data = req.body;

        let result = runner.optimizer(data);

        return res.send(result);

    }catch(err){
        console.error(err);
        return res.status(500).send('Server error.');
    }


});

const PORT = 3000;
app.listen(PORT, () => console.log(`Server listening port ${PORT}.`));

my PM2 ecosystem file is:

module.exports = {
  apps : [{
    name: "Asset Optimizer",
    script: "server.js",
    watch: true,
    ignore_watch: ["optimizer", "outputData", "Rplots.pdf", "nodeSender.js", "errorLog"],
    instances: "max",
    autorestart: true,
    max_memory_restart: "1G",
    exec_mode: "cluster",
    watch_options:{
      "followSymlinks": false
      }
    }
  ]}

I start the server using pm2 start ecosystem.config.js

everything works just fine, but this queue issue is making me crazy. I've tried many many dirty approaches, including splitting routes, splitting servers. no success whatsoever.

Even if you don´t know the answer to this, please give me some ideas on how to overcome this problem. Thank you very much.

UPDATE

I found out that the problem is with the Round Robin policy that is not setup by default in windows.
How do I activate it in pm2?
I've managed to make it work with the native node cluster module using:


const cluster = require('cluster');
const numCPUs = require('os').cpus().length;

cluster.schedulingPolicy = cluster.SCHED_RR;

But once I run the server using PM2, it no longer works.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with server.js and ecosystem.config.js, then inspect PM2's cluster-mode startup and the Node.js schedulingPolicy behavior on Windows. Reproduce the alternating queued requests with instances set to max, and consider the issue resolved when PM2 provides or documents a way to enable round-robin scheduling and requests are distributed across free workers.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js
Domain
devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.