elastic / elastic/elastic-package
benchmarking: mapping for processors is wrong
- Dominant language
- Go
- Stars
- 72
- Forks
- 141
- Avg merge
- 19h 42m
- Merged PRs (30d)
- 55
Description
When running benchmarks, the mapping between the processors listed in the output and the actual source appears to be broken.
```
$ git rev-parse HEAD
1595cc1388bf5054c47d5828535ff772944804df
$ pwd
…/github.com/elastic/integrations/packages/sentinel_one_cloud_funnel
$ elastic-package benchmark pipeline -d event
Run pipeline benchmarks for the package
--- Benchmark results for package: sentinel_one_cloud_funnel - START ---
╭─────────────────────────╮
│ parameters │
├──────────────────┬──────┤
│ source_doc_count │ 16 │
│ doc_count │ 1000 │
╰──────────────────┴──────╯
╭───────────────────────────╮
│ pipeline_performance │
├─────────────────┬─────────┤
│ processing_time │ 0.69s │
│ eps │ 1440.92 │
╰─────────────────┴─────────╯
╭────────────────────────────────────────╮
│ procs_by_total_time │
├───────────────────────────────┬────────┤
│ dot_expander @ default.yml:52 │ 19.31% │
│ json @ default.yml:34 │ 4.61% │
│ remove @ default.yml:2778 │ 3.89% │
│ script @ default.yml:2849 │ 1.73% │
│ date @ default.yml:87 │ 1.44% │
│ date @ default.yml:533 │ 1.44% │
│ date @ default.yml:1723 │ 1.30% │
│ gsub @ default.yml:20 │ 1.01% │
│ script @ default.yml:2770 │ 1.01% │
│ script @ pipeline-dns.yml:21 │ 0.72% │
╰───────────────────────────────┴────────╯
╭──────────────────────────────────────────╮
│ procs_by_avg_time_per_doc │
├───────────────────────────────┬──────────┤
│ dot_expander @ default.yml:52 │ 134µs │
│ json @ default.yml:34 │ 32µs │
│ remove @ default.yml:2778 │ 27µs │
│ script @ pipeline-dns.yml:21 │ 19.841µs │
│ script @ default.yml:2849 │ 12µs │
│ date @ default.yml:533 │ 10.66µs │
│ date @ default.yml:87 │ 10µs │
│ date @ default.yml:1723 │ 9µs │
│ script @ default.yml:2770 │ 7.462µs │
│ gsub @ default.yml:20 │ 7µs │
╰───────────────────────────────┴──────────╯
--- Benchmark results for package: sentinel_one_cloud_funnel - END ---
Done
```
But to take one example, the `dot_expander` since it's unique and near the head of the file, it is not at line 52, it's at [44](https://github.com/elastic/integrations/blob/1595cc1388bf5054c47d5828535ff772944804df/packages/sentinel_one_cloud_funnel/data_stream/event/elasticsearch/ingest_pipeline/default.yml#L44).
Internal discussion has raised the possibility that this is due to the presence of reroute rules in the package, something that was found to be an issue in the pipeline test runner (ref:#1647).
Another thing that would help here is to not only identify the processor by its line number, but to also use its tag if it exists.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the `elastic-package benchmark pipeline -d event` entry point and reproduce the output against `packages/sentinel_one_cloud_funnel/data_stream/event/elasticsearch/ingest_pipeline/default.yml`. Compare reported processor lines with the processors in `default.yml`, including the `dot_expander` location, and investigate whether reroute rules cause the offset described in ref:#1647. Done means processor mappings are accurate and processor tags are shown when available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100