yiisoft / yiisoft/yii2-queue

[Redis]When the number of tasks in the queue exceeds 10,000, the speed is slow and errors may occur.

Open
#312 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status:to be verified type:bug
Dominant language
PHP
Stars
1.1k
Forks
285
Avg merge
5d 3h
Merged PRs (30d)
2

Description

What steps will reproduce the problem?

I am using a redis queue.
When the number of tasks in the queue exceeds 10,000, the worker gets the task very slowly, and may also encounter the following error:

PHP Warning 'yii\base\ErrorException' with message 'A non-numeric value encountered'

in /var/www/rho.one/vendor/yiisoft/yii2-queue/src/drivers/redis/Queue.php:153

Stack trace:
#0 /var/www/rho.one/vendor/yiisoft/yii2-queue/src/cli/Queue.php(117): yii\queue\redis\Queue->yii\queue\redis\{closure}()
#1 /var/www/rho.one/vendor/yiisoft/yii2-queue/src/cli/Queue.php(117): ::call_user_func:{/var/www/rho.one/vendor/yiisoft/yii2-queue/src/cli/Queue.php:117}()
#2 /var/www/rho.one/vendor/yiisoft/yii2-queue/src/drivers/redis/Queue.php(68): yii\queue\redis\Queue->runWorker()
#3 /var/www/rho.one/vendor/yiisoft/yii2-queue/src/drivers/redis/Command.php(76): yii\queue\redis\Queue->run()
#4 /var/www/rho.one/vendor/yiisoft/yii2/base/InlineAction.php(57): yii\queue\redis\Command->actionListen()
#5 /var/www/rho.one/vendor/yiisoft/yii2/base/InlineAction.php(57): ::call_user_func_array:{/var/www/rho.one/vendor/yiisoft/yii2/base/InlineAction.php:57}()
#6 /var/www/rho.one/vendor/yiisoft/yii2/base/Controller.php(157): yii\base\InlineAction->runWithParams()
#7 /var/www/rho.one/vendor/yiisoft/yii2/console/Controller.php(148): yii\queue\redis\Command->runAction()
#8 /var/www/rho.one/vendor/yiisoft/yii2/base/Module.php(528): yii\queue\redis\Command->runAction()
#9 /var/www/rho.one/vendor/yiisoft/yii2/console/Application.php(180): yii\console\Application->runAction()
#10 /var/www/rho.one/vendor/yiisoft/yii2/console/Application.php(147): yii\console\Application->runAction()
#11 /var/www/rho.one/vendor/yiisoft/yii2/base/Application.php(386): yii\console\Application->handleRequest()
#12 /var/www/rho.one/yii(27): yii\console\Application->run()
#13 {main}

The problem comes from here:
https://github.com/yiisoft/yii2-queue/blob/master/src/drivers/redis/Queue.php#L153

The $ttr variable may not be a number in some cases.

What's expected?

Solve the above errors and improve performance.

What do you get instead?
Additional info
Q A
Yii version 2.0.16.1
PHP version 7.2.14
Operating system Ubuntu 18.04.1
redis 5.0.3

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/drivers/redis/Queue.php at line 153 and reproduce the worker slowdown with more than 10,000 queued tasks. Trace how $ttr is obtained and used, then verify that the non-numeric warning is eliminated and queue processing performance improves.

Written by the indexing model from the issue text.

Assessment

Tech stack
php, redis
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.