[performance-profiler] Reuse dissect position buffers to cut hot-path allocations
- Dominant language
- Go
- Stars
- 12.7k
- Forks
- 5k
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 370
Description
## Hot Path
`(*Dissector).extract` and its callers `(*Dissector).Dissect` / `(*Dissector).DissectConvert` in `libbeat/processors/dissect/dissect.go:57-107` and `libbeat/processors/dissect/dissect.go:117-193` allocate a new `[]position` on every call to `extract`, which shows up as a major allocator hotspot in benchmark profiles.
## Profiling Data
**Before:**
```text
go test ./libbeat/processors/dissect -run '^$' -bench '^BenchmarkDissectComplexStackTraceDegradation$' -benchmem -cpuprofile=/tmp/gh-aw/agent/dissect.before.cpu.prof -memprofile=/tmp/gh-aw/agent/dissect.before.mem.prof -count=5
BenchmarkDissectComplexStackTraceDegradation/ComplexStackTrace-8-4 1274858 929.2 ns/op 808 B/op 5 allocs/op
BenchmarkDissectComplexStackTraceDegradation/ComplexStackTrace-8-4 1256325 930.6 ns/op 808 B/op 5 allocs/op
BenchmarkDissectComplexStackTraceDegradation/ComplexStackTrace-8-4 1290156 941.9 ns/op 808 B/op 5 allocs/op
BenchmarkDissectComplexStackTraceDegradation/ComplexStackTrace-8-4 1300606 941.5 ns/op 808 B/op 5 allocs/op
BenchmarkDissectComplexStackTraceDegradation/ComplexStackTrace-8-4 1282668 927.9 ns/op 808 B/op 5 allocs/op
```
`go tool pprof -top -alloc_space /tmp/gh-aw/agent/dissect.before.mem.prof` excerpt:
```text
flat flat% sum% cum cum%
35.57GB 60.07% 60.07% 35.57GB 60.07% github.com/elastic/beats/v7/libbeat/processors/dissect.normalField.Apply
13.04GB 22.02% 82.09% 48.60GB 82.09% github.com/elastic/beats/v7/libbeat/processors/dissect.(*Dissector).resolve
10.59GB 17.89% 100.00% 10.59GB 17.89% github.com/elastic/beats/v7/libbeat/processors/dissect.(*Dissector).extract
```
## Proposed Change
Use a caller-owned positions buffer in `extract` instead of allocating internally, and use a stack-backed fast path for common field counts in callers.
```diff
diff --git a/libbeat/processors/dissect/dissect.go b/libbeat/processors/dissect/dissect.go
@@ -59,8 +59,15 @@ func (d *Dissector) Dissect(s string) (Map, error) {
- positions, err := d.extract(s)
+ var positions positions
+ if len(d.parser.fields) <= 16 {
+ var stackPositions [16]position
+ positions = stackPositions[:len(d.parser.fields)]
+ } else {
+ positions = make([]position, len(d.parser.fields))
+ }
- if err != nil {
+ if err := d.extract(s, positions); err != nil {
return nil, err
}
@@ -100,8 +114,7 @@ func (d *Dissector) Raw() string {
-func (d *Dissector) extract(s string) (positions, error) {
- positions := make([]position, len(d.parser.fields))
+func (d *Dissector) extract(s string, positions positions) error {
@@
- return positions, nil
+ return nil
}
```
## Results
**After:**
```text
go test ./libbeat/processors/dissect -run '^$' -bench '^BenchmarkDissectComplexStackTraceDegradation$' -benchmem -cpuprofile=/tmp/gh-aw/agent/dissect.after.cpu.prof -memprofile=/tmp/gh-aw/agent/dissect.after.mem.prof -count=5
BenchmarkDissectComplexStackTraceDegradation/ComplexStackTrace-8-4 1402860 844.8 ns/op 664 B/op 4 allocs/op
BenchmarkDissectComplexStackTraceDegradation/ComplexStackTrace-8-4 1385682 857.8 ns/op 664 B/op 4 allocs/op
BenchmarkDissectComplexStackTraceDegradation/ComplexStackTrace-8-4 1364835 855.8 ns/op 664 B/op 4 allocs/op
BenchmarkDissectComplexStackTraceDegradation/ComplexStackTrace-8-4 1381454 857.0 ns/op 664 B/op 4 allocs/op
BenchmarkDissectComplexStackTraceDegradation/ComplexStackTrace-8-4 1394032 846.2 ns/op 664 B/op 4 allocs/op
```
`go tool pprof -top -alloc_space /tmp/gh-aw/agent/dissect.after.mem.prof` excerpt:
```text
flat flat% sum% cum cum%
39.46GB 74.15% 74.15% 39.46GB 74.15% github.com/elastic/beats/v7/libbeat/processors/dissect.normalField.Apply
13.74GB 25.83% 100.00% 53.20GB 100.00% github.com/elastic/beats/v7/libbeat/processors/dissect.(*Dissector).resolve
```
**Improvement:**
- `ComplexStackTrace-8-4` memory: `808 B/op -> 664 B/op` (**17.8% reduction**)
- `ComplexStackTrace-8-4` allocations: `5 allocs/op -> 4 allocs/op` (**20.0% reduction**)
- `ComplexStackTrace-8-4` time (mean of shown runs): `934.2 ns/op -> 852.3 ns/op` (**8.8% faster**)
This crosses the reporting threshold via memory/allocation improvement on a benchmarked hot path.
## Verification
- `go test ./libbeat/processors/dissect -count=1` passed after the optimization patch in the profiling run.
- Change is internal-only buffer management in dissect parsing; no API or behavior change intended.
## Evidence
- Baseline benchmark command and optimized benchmark command are identical except for code state.
- Before/after outputs are materially different (B/op and allocs/op changed; timing also changed).
- Duplicate check against `/tmp/previous-findings.json`: no matching open issue title for this exact optimization.
---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Performance Profiler](https://github.com/elastic/beats/actions/runs/34607864157)
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Sep 18, 2026, 2:22 PM UTC
Contributor guide
Research direction
Start in libbeat/processors/dissect/dissect.go, especially Dissector.extract and its Dissect/DissectConvert callers. Run the named BenchmarkDissectComplexStackTraceDegradation benchmark with -benchmem, then run go test ./libbeat/processors/dissect -count=1. Done means the positions buffer is caller-owned, behavior remains unchanged, and the benchmark confirms lower allocations and memory use.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100