hyperf / hyperf/hyperf

[BUG] 建议优化Class Hyperf\Metric\Adapter\Prometheus\Redis中的collectSamples方法的性能

Open
#7,253 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
PHP
Stars
6.9k
Forks
1.3k
Avg merge
4h 21m
Merged PRs (30d)
4

Description

在hyperf\metric\src\Adapter\Prometheus\Redis.php中,这个protected function collectSamples(string $metricType): array方法的实现效率太低,对于有海量数据的指标,会导致长时间锁定key,从而导致想要对相关key进行操作时,超时失败,我的修改如下,经过测试性能提升非常显著:
/**
* @throws RedisException
/
protected function collectSamples(string $metricType): array
{
$keys = $this->redis->sMembers($this->getMetricGatherKey($metricType));
sort($keys);
foreach ($keys as $key) {
$key = str_replace($this->redis->_prefix(''), '', $key);
$metastr = $this->redis->hget($key, '__meta');
if(!$metastr) continue;
$sample = array_merge(Json::decode($metastr), ['samples' => []]);
$cursor = NULL;
do {
$raw = $this->redis->hscan($key, $cursor, '', 1000);
unset($raw['__meta']);
foreach ($raw as $k => $value) {
if (count($sample['labelNames'] ?? []) !== count(json_decode($k, true))) {
continue;
}

$sample['samples'][] = [
'name' => $sample['name'],
'labelNames' => [],
'labelValues' => Json::decode($k),
'value' => $value,
];
}
} while ($cursor != 0);

usort($sample['samples'], fn ($a, $b) => strcmp(implode('', $a['labelValues']), implode('', $b['labelValues'])));
$samples[] = $sample;
}

return $samples ?? [];
}

修改的重点是将hgetall改为hscan方法

Contributor guide

Open the contributing guide

Research direction

Start in hyperf/metric/src/Adapter/Prometheus/Redis.php at the protected collectSamples(string $metricType): array method. Compare the current hgetall-based collection with the proposed hscan approach, including metadata handling, cursor iteration, sorting, and label filtering. Done means large metrics no longer hold the Redis key for so long while the collected samples remain correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
php, prometheus, redis
Domain
backend, databases, observability, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.