tarantool / tarantool/metrics

collector.histogram is slow

Open
#461 2 comments 0 reactions 1 assignee View on GitHub

@yngvar-antonsson is already working on this.

Since Sep 13, 2023.

feature
Dominant language
Lua
Stars
40
Forks
28
Avg merge
15h 3m
Merged PRs (30d)
4

Description

collector histogram is just bunch of counters, it is too bad for it to be slow. I've slightly rewrite collectors/histogram to speed it up:

With constant labels for each observe

127.0.0.1:3301> clock.bench(function() local lbls = {method='api.users.get',ok='true'} local prev = clock.time() for _ = 1, 1e6 do api_timings_hist:observe(clock.time()-prev, lbls) prev = clock.time() end end)
---
- - 16.211009
...

127.0.0.1:3301> clock.bench(function() local lbls = {method='api.users.get',ok='true'} local prev = clock.time() for _ = 1, 1e6 do api_timings_fast:observe(clock.time()-prev, lbls) prev = clock.time() end end)
---
- - 1.884366
...

Without labels on observe

127.0.0.1:3301> clock.bench(function() local prev = clock.time() for _ = 1, 1e6 do api_timings_hist:observe(clock.time()-prev) prev = clock.time() end end)
---
- - 10.5773
...
127.0.0.1:3301> clock.bench(function() local prev = clock.time() for _ = 1, 1e6 do api_timings_fast:observe(clock.time()-prev) prev = clock.time() end end)
---
- - 1.810581
...

Source code of FastHistogram is available on my gist: https://gist.github.com/ochaton/305e4668b41e2721b82b92217d74c9ba

Initialisation

local metrics = require 'metrics'
local metrics_fast_hist = require 'metrics.fast'

metrics.set_global_labels({
	app_name = 'bench',
	instance_name = 'bench_001',
})

local api_timings_fast = metrics_fast_hist:new('api_call', 'Histogram of API call',nil, {})
local api_timings_hist = metrics.histogram('api_call_hist', 'Histogram of API call',nil, {})

rawset(_G, 'api_timings_hist', api_timings_hist)
rawset(_G, 'api_timings_fast', api_timings_fast)

@yngvar-antonsson please take a look, and me you'll suggest better approach to increase perfomance of metrics/histogram

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.