netdata / netdata/netdata

[Feat]: carts for mariadb circular buffers

Open
#17,656 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature request needs triage
Dominant language
Go
Stars
80.6k
Forks
6.6k
Avg merge
15h 36m
Merged PRs (30d)
304

Description

Problem

We use mariadb with galera. To investigate and tune it's state transfre capability, we need to track utilization of two circular buffers, namely Gcache and Redo log (innodb_log). We need to know how much history buffers are able to hold in our setup.

Description

As buffers are circular, they act like LIFO, so once the buffer is completely utilized, adding new data purges oldest records.
For each buffer we would need three charts:

  • circular buffer size
  • total amount of bytes written into the buffer
  • time since the moment the buffer is completely utilized first since mariadbd start-up.

In order not to overload mariadb with reading parameters, it's enough to have these values checked once a minute.
At the moment Netdata already collects wsrep_replicated_bytes per-second delta (shown on chart mysql.galera_bytes and Innodb log buffer per-second delta is present in mysql.innodb_io, but we need absolute values also.

Importance

really want

Value proposition
  • For Gcache:
  1. buffer size on disk
  2. amount of replicated data
  3. time since first overflow
  • For Innodb log buffer:
  1. buffer size on disk
  2. amount of data written
  3. time since first overflow
Proposed implementation
  • For gcache buffer:
  1. size can be obtained as a field gcache.keep_pages_size from the wsrep_provider_options variable (that can be read as SHOW VARIABLES LIKE 'wsrep_provider_options'\G).
  2. amount of replicated data can be obtained from SHOW STATUS LIKE 'wsrep_replicated_bytes'; output directly.
  3. time since first overflow is 0 until the moment of (wsrep_replicated_bytes = gcache.keep_pages_size), than it should be the difference between that moment and current time.
  • For Innodb log buffer:
  1. size can be read from system variable SHOW STATUS LIKE 'innodb_log_file_size'; directly
  2. amount of written data can be read from system variable SHOW STATUS LIKE 'Innodb_os_log_written';
  3. time since first overflow is 0 until the moment of (innodb_log_file_size = Innodb_os_log_written), than it should be the difference between that moment and current time.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the existing mysql.galera_bytes and mysql.innodb_io charts and the collector queries for wsrep_replicated_bytes and Innodb_os_log_written. Compare those entry points with the requested SHOW variables and determine where once-per-minute collection and overflow timing belong. Done means both Gcache and InnoDB log buffers expose size, written bytes, and time since first overflow with suitable charts and tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
mariadb, mysql
Domain
databases, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.