Performance Regression or Improvement: sideinpts_python_batch_1gb_1kb_10workers_1000window_99key_percent_dict:runtime
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
Performance change found in the
test: `sideinpts_python_batch_1gb_1kb_10workers_1000window_99key_percent_dict` for the metric: `runtime`.
For more information on how to triage the alerts, please look at
`Triage performance alert issues` section of the [README](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/testing/analyzers/README.md#triage-performance-alert-issues).
`Test description:` python | Side Input | 1 GB dictionary, 99% of keys, 1000 fixed windows
Test Link - https://github.com/apache/beam/blob/5e38decf9e723a385057131b01bbd33d8c60bda3/.test-infra/jenkins/job_LoadTests_SideInput_Python.groovy#L133
Test Dashboard - http://metrics.beam.apache.org/d/-E9aGlFGk/side-input-load-tests?orgId=1&from=now-90d&to=now&viewPanel=9
```
timestamp: Thu Sep 3 20:17:32 2026, metric_value: 1808.00
timestamp: Wed Sep 2 19:47:35 2026, metric_value: 1512.00
timestamp: Tue Sep 1 19:55:57 2026, metric_value: 1275.00
timestamp: Mon Aug 31 19:41:16 2026, metric_value: 1816.00 <---- Anomaly
timestamp: Sun Aug 30 19:04:58 2026, metric_value: 827.00
timestamp: Sat Aug 29 19:02:56 2026, metric_value: 1097.00
timestamp: Fri Aug 28 23:01:31 2026, metric_value: 980.00
timestamp: Thu Aug 27 22:52:17 2026, metric_value: 870.00
timestamp: Wed Aug 26 20:06:49 2026, metric_value: 955.00
timestamp: Tue Aug 25 19:18:13 2026, metric_value: 1217.00
timestamp: Mon Aug 24 19:33:43 2026, metric_value: 1004.00
timestamp: Sun Aug 23 19:04:30 2026, metric_value: 970.00
timestamp: Sat Aug 22 19:29:03 2026, metric_value: 971.00
timestamp: Fri Aug 21 19:09:20 2026, metric_value: 1040.00
```
Contributor guide
Research direction
Start with the “Triage performance alert issues” section in sdks/python/apache_beam/testing/analyzers/README.md, then inspect the test definition at .test-infra/jenkins/job_LoadTests_SideInput_Python.groovy#L133 and the linked dashboard. Compare the anomalous runtime with nearby runs and determine whether the alert represents a regression; done means the cause or required follow-up is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100