elastic / elastic/beats

[performance-profiler] Optimize multiline '^=[A-Z]+' matcher by avoiding regexp fallback

Open
#52,482 1 comment 0 reactions 0 assignees View on GitHub
needs_team
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

## Hot Path
`libbeat/reader/multiline/pattern.go:201` and `libbeat/reader/multiline/pattern.go:231` call `pr.pred(...)` for each line while grouping multiline events.

For the RabbitMQ-like benchmark pattern (`^=[A-Z]+`), matcher compilation currently falls through to regexp in `libbeat/common/match/compile.go:78-81` (default branch with `regexp.Compile`).

## Profiling Data
**Before:**
```
$ go test ./libbeat/reader/multiline -run '^$' -bench '^BenchmarkPatternReaderRabbitMQLike$' -benchmem -benchtime=1s -count=5
BenchmarkPatternReaderRabbitMQLike-4 1552 819221 ns/op 60.62 MB/s 560996 B/op 5677 allocs/op
BenchmarkPatternReaderRabbitMQLike-4 1354 832418 ns/op 59.66 MB/s 560873 B/op 5677 allocs/op
BenchmarkPatternReaderRabbitMQLike-4 1544 809218 ns/op 61.37 MB/s 561122 B/op 5677 allocs/op
BenchmarkPatternReaderRabbitMQLike-4 1382 794681 ns/op 62.50 MB/s 560712 B/op 5677 allocs/op
BenchmarkPatternReaderRabbitMQLike-4 1550 810539 ns/op 61.27 MB/s 560736 B/op 5677 allocs/op
```

## Proposed Change
Add a specialized matcher compile path for the anchored uppercase-prefix pattern (`^=[A-Z]+`) so this case does not route through `regexp.Compile(...)`.

Prototype (local validation) added:
- Pattern detection in `libbeat/common/match/cmp.go`
- Compile branch in `libbeat/common/match/compile.go`
- Fast matcher implementation in `libbeat/common/match/matchers.go`
- Matcher type assertion and behavior checks in `libbeat/common/match/matcher_test.go`

Prototype matcher logic:
```go
func (m *prefixUpperASCIIPlusMatcher) Match(in []byte) bool {
if len(in) <= len(m.prefix) || !bytes.Equal(in[:len(m.prefix)], m.prefix) {
return false
}
c := in[len(m.prefix)]
return c >= 'A' && c <= 'Z'
}
```

## Results
**After:**
```
$ go test ./libbeat/reader/multiline -run '^$' -bench '^BenchmarkPatternReaderRabbitMQLike$' -benchmem -benchtime=1s -count=5
BenchmarkPatternReaderRabbitMQLike-4 1964 634115 ns/op 78.32 MB/s 555533 B/op 5676 allocs/op
BenchmarkPatternReaderRabbitMQLike-4 2025 605796 ns/op 81.98 MB/s 555528 B/op 5676 allocs/op
BenchmarkPatternReaderRabbitMQLike-4 1990 622061 ns/op 79.84 MB/s 555531 B/op 5676 allocs/op
BenchmarkPatternReaderRabbitMQLike-4 1903 614444 ns/op 80.83 MB/s 555534 B/op 5676 allocs/op
BenchmarkPatternReaderRabbitMQLike-4 2034 610162 ns/op 81.39 MB/s 555531 B/op 5676 allocs/op
```

**Improvement:**
- Mean time: ~813,215 ns/op -> ~617,316 ns/op (**~24.1% faster**)
- Mean memory: ~560,888 B/op -> ~555,531 B/op (~0.95% lower)
- Allocs/op: 5677 -> 5676

## Verification
- `go test ./libbeat/common/match -run '^TestMatchers$'` passes with the prototype matcher path.
- Change is behavior-preserving for the benchmarked pattern semantics (`^=[A-Z]+`): anchored prefix plus one-or-more uppercase bytes.

## Evidence
Commands used:
```
go test ./libbeat/reader/multiline -run '^$' -bench '^BenchmarkPatternReaderRabbitMQLike$' -benchmem -benchtime=1s -count=5
go test ./libbeat/common/match -run '^TestMatchers$'
```

Duplicate check performed against `/tmp/previous-findings.json` and open-issue search; no matching open perf-profiler issue for this matcher path was found.

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Performance Profiler](https://github.com/elastic/beats/actions/runs/31112403058)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Aug 13, 2026, 3:07 PM UTC

Contributor guide

Open the contributing guide

Research direction

Start with libbeat/common/match/cmp.go, compile.go, and matchers.go, then inspect matcher_test.go and the existing calls in libbeat/reader/multiline/pattern.go. Run TestMatchers and BenchmarkPatternReaderRabbitMQLike with the commands in the issue. Done means the anchored uppercase-prefix case preserves the stated behavior and shows the reported performance improvement without breaking the match tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.