prometheus / prometheus/prometheus
ST: Optimize CPU/memory use for OM1.0 Text CT/ST parsing (`st-storage` and `created-timestamp-zero-ingestion` features)
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 66.1k
- Forks
- 10.8k
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 131
Description
Proposal
As observed accidently on main and without any feature flag we use a lot of memory for CreatedTimestamp call for OM text. This is .due to deep coping of parser on every CreatedTimestamp, so technically per every series (!). This was a known naive implementation we decided to accept when iterating on this feature in https://github.com/prometheus/prometheus/pull/14356
This issue is to minimize that memory use. We already discussed this, but to put it here:
- We should see if we can reuse existing parser (not copy it).
- If we have to copy, copy once and reuse a second parser (e.g. reset it's positions).
IMPORTANT: This is only relevant to OpenMetrics 1.0. We designed OM 2 to not have that problem.
Acceptance Criteria
- The additional memory use per OM Text scrape with CT feature is minimal (~within +5% max)
Related to https://github.com/prometheus/prometheus/issues/14217
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in model/textparse/openmetricsparse.go at the CreatedTimestamp path and review how the OpenMetrics 1.0 parser is copied for each series. Measure memory for an OM text scrape with the created-timestamp feature, then verify the change keeps additional use within the stated roughly 5% limit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, performance
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100