[refactor] Code duplication findings (2026-05-22)
- Dominant language
- Go
- Stars
- 12.7k
- Forks
- 5k
- Avg merge
- 2d 54m
- Merged PRs (30d)
- 381
Description
## Summary
- Files analyzed: 200 (largest non-test, non-generated Go/Python files)
- Functions cataloged: 4182
- Clusters with issues: 2
## Findings
### 1. Near-identical metric processing pipeline is duplicated in OpenMetrics and Prometheus helpers
**Occurrences:**
- `metricbeat/helper/openmetrics/openmetrics.go:129-243` (`(*openmetrics).ProcessMetrics`)
- `metricbeat/helper/prometheus/prometheus.go:139-254` (`(*prometheus).ProcessMetrics`)
**Evidence:**
Both functions implement the same end-to-end control flow and data transformations, including the same event map initialization, metric-family loops, mapped-label conversion, `infoMetrics` merge pass, and return pattern.
Quoted matching code includes:
- `eventsMap := map[string]mapstr.M{}` and `infoMetrics := []*infoMetricData{}`
- `for _, family := range families {` and nested `for _, metric := range family.GetMetric() {`
- `for k, v := range info.Labels.Flatten() {` with `value, err := event.GetValue(k)` comparison
- `return events, nil`
**What is wrong:**
Two ~100-line implementations are maintained separately for effectively the same metric processing behavior.
**Why it matters:**
Bug fixes and behavior changes can drift between the OpenMetrics and Prometheus paths, creating inconsistent metric mapping semantics and avoidable maintenance overhead.
**Suggested fix:**
Extract a shared internal metric-processing helper (parameterized by family/metric access accessors or small adapter interfaces), and keep `ProcessMetrics` in each package as thin wrappers.
### 2. `retryablehttp` logger shim is duplicated in four Filebeat input/provider implementations
**Occurrences:**
- `x-pack/filebeat/input/cel/input.go:1682-1692` (`retryLog`, `newRetryLog`, methods)
- `x-pack/filebeat/input/salesforce/input.go:976-986` (`retryLog`, `newRetryLog`, methods)
- `x-pack/filebeat/input/entityanalytics/provider/jamf/jamf.go:308-318` (`retryLog`, `newRetryLog`, methods)
- `x-pack/filebeat/input/entityanalytics/provider/okta/okta.go:323-333` (`retryLog`, `newRetryLog`, methods)
**Evidence:**
All four files contain the same adapter type and method set:
- `type retryLog struct{ log *logp.Logger }`
- `return &retryLog{log: log.Named("retryablehttp").WithOptions(zap.AddCallerSkip(1))}`
- `Error/Info/Debug/Warn` forwarding methods to `*logp.Logger` (`Errorw`, `Infow`, `Debugw`, `Warnw`).
**What is wrong:**
The same non-trivial adapter is copy-pasted across four code paths.
**Why it matters:**
Any change to logging behavior (caller-skip policy, fields, method mapping) requires synchronized edits in four places and risks behavioral drift between inputs/providers.
**Suggested fix:**
Move the shim into a shared internal helper package (for example under `x-pack/filebeat/input/internal/...`) and reuse a single implementation from CEL, Salesforce, Jamf, and Okta paths.
## Suggested Actions
- [ ] Extract and adopt a shared metric processing helper for OpenMetrics and Prometheus `ProcessMetrics` implementations.
- [ ] Extract and adopt a shared `retryablehttp` logger shim helper for Filebeat HTTP-based inputs/providers.
## Analysis Metadata
- Serena tools used: unavailable in this runtime (fallback used: scoped file inventory + direct source verification)
- Analysis date: 2026-05-22T13:06:44Z
> [!NOTE]
>
> 🔒 Integrity filter blocked 17 items
>
> The following items were blocked because they don't meet the GitHub integrity level.
>
> - [#43899](https://github.com/elastic/beats/issues/43899) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#44484](https://github.com/elastic/beats/issues/44484) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#32621](https://github.com/elastic/beats/issues/32621) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#27050](https://github.com/elastic/beats/issues/27050) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#12821](https://github.com/elastic/beats/issues/12821) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#23706](https://github.com/elastic/beats/issues/23706) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#5396](https://github.com/elastic/beats/issues/5396) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#206](https://github.com/elastic/beats/issues/206) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#6867](https://github.com/elastic/beats/issues/6867) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#49035](https://github.com/elastic/beats/issues/49035) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#47267](https://github.com/elastic/beats/issues/47267) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#31869](https://github.com/elastic/beats/issues/31869) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#23616](https://github.com/elastic/beats/issues/23616) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#25429](https://github.com/elastic/beats/issues/25429) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#18815](https://github.com/elastic/beats/issues/18815) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - [#25391](https://github.com/elastic/beats/issues/25391) `search_issues`: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
> - ... and 1 more item
>
> To allow these resources, lower `min-integrity` in your GitHub frontmatter:
>
> ```yaml
> tools:
> github:
> min-integrity: approved # merged | approved | unapproved | none
> ```
>
>
---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Code Duplication Detector](https://github.com/elastic/beats/actions/runs/26289425730)
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on May 29, 2026, 1:15 PM UTC
Contributor guide
Assessment
This issue has not been assessed yet.