redpanda-data / redpanda-data/redpanda

Expose host/container THP stats

Open
#14,136 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

kind/enhance performance
Dominant language
C++
Stars
12.6k
Forks
792
PR merge metrics
No merged PRs in 30d

Description

Who is this for and what problem do they have today?

Perf people who want to see the THP status.

What are the success criteria?

A metric which exposes the THP "state" of the cluster, i.e., the fraction of the main memory region which is backed by THP. This value is interesting both at startup (where it will reflect mostly the existing state of VA fragmentation on the host) and also over the lifetime of the process as it may change, e.g., as things mprotect regions which will split the VMA and huge pages in general (ss::thread does this as @rockwotj points out).

Why is solving this problem impactful?

Calling it "impactful" might be a stretch, but I personally would enjoy looking at this metric before I go to bed.

Additional notes

Probably we just grab it at scrap time. It involves parsing the /proc/maps and identifying the seastar map and checking how much of it is hugepages. If this ends up being slow-ish maybe we limit it to once every N seconds (if the user scrapes more frequently than that, the metric simply won't change).

Not sure if this info is available in k8s.

JIRA Link: CORE-1502

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source file or test is named. Start by locating the existing metric exposition and scrape path, then inspect how /proc/maps can identify the seastar map and its THP-backed portion for host and container environments. Done means exposing a metric for the fraction of the main memory region backed by THP, with an appropriate scrape-time or caching strategy.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, linux
Domain
observability, operating-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.