elastic / elastic/beats

[performance-profiler] Reuse OTel filestorage Each decoder to reduce allocations

Open Beginner friendly
#51,982 1 comment 0 reactions 0 assignees View on GitHub
Team:Elastic-Agent-Data-Plane
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 2h
Merged PRs (30d)
364

Description

## Hot Path
`libbeat/statestore/backend/otelstorage/store_from_client.go:96-112`, specifically `(*storeFromClient).Each` line 102. The `Walk` callback allocates a new `*jsonValueDecoder` for every stored key, even though the `ValueDecoder` contract says it is invalid after the callback returns (`libbeat/statestore/backend/backend.go:65-70`).

## Profiling Data
**Before:**
```
go test -run=^$ -bench=BenchmarkCRUD/otel_file_storage/Each$ -benchmem -benchtime=1s -count=5 ./libbeat/statestore/backend/benchmarks
BenchmarkCRUD/otel_file_storage/Each: median 683078 ns/op, 485233 B/op, 20021 allocs/op
```
A reproduced five-run baseline on the current checkout measured 664225-667674 ns/op with the same 485233-485234 B/op and 20021 allocs/op. The allocation profile attributed 576 MB (34.99% flat) to `storeFromClient.Each.func1`.

## Proposed Change
Create one `jsonValueDecoder` before calling `walker.Walk`, assign its `raw` field to the current `value` inside the callback, and pass that decoder to `fn`. This is behavior-preserving because the backend contract explicitly invalidates the decoder after each callback.

## Results
**After (same benchmark command):**
```
BenchmarkCRUD/otel_file_storage/Each: median 578239 ns/op, 245265 B/op, 10022 allocs/op
```
**Improvement:** 15.35% lower ns/op, 49.45% fewer bytes/op, and 49.94% fewer allocations/op.

## Verification
- `go test ./libbeat/statestore/backend/otelstorage` passed, including `TestStoreFromClient_Each`.
- The temporary optimization was reverted after measurement; no repository changes remain.
- The decoder remains valid for all repeated `Decode` calls within each callback and is intentionally invalidated when the callback returns.

## Evidence
- Benchmark: `libbeat/statestore/backend/benchmarks/benchmarks_test.go:137-149` preloads 10,000 keys and repeatedly calls `Each`.
- Implementation: `libbeat/statestore/backend/otelstorage/store_from_client.go:101-104` allocates the decoder per key.
- Contract: `libbeat/statestore/backend/backend.go:65-70`.
- Searches found no issue or pull request specifically tracking `storeFromClient`/`jsonValueDecoder` reuse. Existing OTel filestorage benchmark work in PR #50130 documents `Each` overhead but does not address this allocation.

Suggested action: add a focused regression benchmark or retain the existing benchmark while reusing the decoder instance in `Each`.

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Performance Profiler](https://github.com/elastic/beats/actions/runs/29423943756)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Jul 22, 2026, 2:47 PM UTC

Contributor guide

Open the contributing guide

Research direction

Start in libbeat/statestore/backend/otelstorage/store_from_client.go:101-104 and read the ValueDecoder contract in libbeat/statestore/backend/backend.go:65-70. Run BenchmarkCRUD/otel_file_storage/Each from libbeat/statestore/backend/benchmarks, then run go test ./libbeat/statestore/backend/otelstorage. Done means Each preserves callback decoding while reducing its measured allocations and passing the existing tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, performance, testing
Issue type
Refactor
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.