devloperdevesh / devloperdevesh/FaultPlane
[Feat/Infra]: Build a Lock-Free Ring Buffer Memory Allocator inside internal/storage/ using sync/atomic Flags
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 2
- Forks
- 1
- Avg merge
- 14m
- Merged PRs (30d)
- 57
Description
Problem
High-frequency logging setups and scraping metrics collectors perform dynamic allocation routines during rapid parallel loops. Locking structural counters during parallel scrape windows triggers tracking jitter and minor data leaks inside telemetry nodes.
Proposed Solution
Refactor the metric pipeline tracking layers in internal/telemetry/collector.go to run on top of an independent Vectorized Lock-Free Metrics Collector Layer.
- Store operational event counters using fixed arrays of primitive 64-bit integer values mapped explicitly to individual telemetry dimensions.
- Update tracking counters exclusively utilizing low-overhead atomic addition routines (
atomic.AddInt64). - Stream compiled tracking data segments down to Prometheus text scrapers utilizing optimized multi-buffer vector writers (
unix.Writev) to maximize disk I/O throughput.
Alternatives
- Utilizing standard Prometheus telemetry libraries, which provide convenient abstraction APIs but introduce object allocation loops inside critical hot routing connection pipelines.
Use Case
Maintains flawless system-wide observability statistics visibility under intense stress workloads without introducing a single byte of memory heap allocation overhead.
Additional Notes
Expose these optimized telemetry output channels cleanly alongside standard dashboards definitions mapped within deployments configs.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with internal/telemetry/collector.go and trace how metric pipeline tracking currently works, then review the proposed sync/atomic counters and unix.Writev output path. Done means the requested lock-free telemetry layer, optimized Prometheus output channels, and deployment dashboard definitions are implemented without heap allocation in the stated hot paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, prometheus
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100