[Redis]When the number of tasks in the queue exceeds 10,000, the speed is slow and errors may occur.
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 1.1k
- Forks
- 285
- Avg merge
- 5d 3h
- Merged PRs (30d)
- 2
Description
What steps will reproduce the problem?
I am using a redis queue.
When the number of tasks in the queue exceeds 10,000, the worker gets the task very slowly, and may also encounter the following error:
PHP Warning 'yii\base\ErrorException' with message 'A non-numeric value encountered'
in /var/www/rho.one/vendor/yiisoft/yii2-queue/src/drivers/redis/Queue.php:153
Stack trace:
#0 /var/www/rho.one/vendor/yiisoft/yii2-queue/src/cli/Queue.php(117): yii\queue\redis\Queue->yii\queue\redis\{closure}()
#1 /var/www/rho.one/vendor/yiisoft/yii2-queue/src/cli/Queue.php(117): ::call_user_func:{/var/www/rho.one/vendor/yiisoft/yii2-queue/src/cli/Queue.php:117}()
#2 /var/www/rho.one/vendor/yiisoft/yii2-queue/src/drivers/redis/Queue.php(68): yii\queue\redis\Queue->runWorker()
#3 /var/www/rho.one/vendor/yiisoft/yii2-queue/src/drivers/redis/Command.php(76): yii\queue\redis\Queue->run()
#4 /var/www/rho.one/vendor/yiisoft/yii2/base/InlineAction.php(57): yii\queue\redis\Command->actionListen()
#5 /var/www/rho.one/vendor/yiisoft/yii2/base/InlineAction.php(57): ::call_user_func_array:{/var/www/rho.one/vendor/yiisoft/yii2/base/InlineAction.php:57}()
#6 /var/www/rho.one/vendor/yiisoft/yii2/base/Controller.php(157): yii\base\InlineAction->runWithParams()
#7 /var/www/rho.one/vendor/yiisoft/yii2/console/Controller.php(148): yii\queue\redis\Command->runAction()
#8 /var/www/rho.one/vendor/yiisoft/yii2/base/Module.php(528): yii\queue\redis\Command->runAction()
#9 /var/www/rho.one/vendor/yiisoft/yii2/console/Application.php(180): yii\console\Application->runAction()
#10 /var/www/rho.one/vendor/yiisoft/yii2/console/Application.php(147): yii\console\Application->runAction()
#11 /var/www/rho.one/vendor/yiisoft/yii2/base/Application.php(386): yii\console\Application->handleRequest()
#12 /var/www/rho.one/yii(27): yii\console\Application->run()
#13 {main}
The problem comes from here:
https://github.com/yiisoft/yii2-queue/blob/master/src/drivers/redis/Queue.php#L153
The $ttr variable may not be a number in some cases.
What's expected?
Solve the above errors and improve performance.
What do you get instead?
Additional info
| Q | A |
|---|---|
| Yii version | 2.0.16.1 |
| PHP version | 7.2.14 |
| Operating system | Ubuntu 18.04.1 |
| redis | 5.0.3 |
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/drivers/redis/Queue.php at line 153 and reproduce the worker slowdown with more than 10,000 queued tasks. Trace how $ttr is obtained and used, then verify that the non-numeric warning is eliminated and queue processing performance improves.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php, redis
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100