Performance Regression or Improvement: test_cloudml_benchmark_criteo_10GB-runtime_sec:runtime_sec
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
Performance change found in the
test: `test_cloudml_benchmark_criteo_10GB-runtime_sec` for the metric: `runtime_sec`.
For more information on how to triage the alerts, please look at
`Triage performance alert issues` section of the [README](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/testing/analyzers/README.md#triage-performance-alert-issues).
`Test description:` TFT Criteo test on 10 GB data.
Test link - https://github.com/apache/beam/blob/42d0a6e3564d8b9c5d912428a6de18fb22a13ac1/sdks/python/apache_beam/testing/benchmarks/cloudml/cloudml_benchmark_test.py#LL104C7-L104C41
```
timestamp: Wed Aug 5 23:21:10 2026, metric_value: 2800.29
timestamp: Tue Aug 4 23:22:17 2026, metric_value: 2808.33
timestamp: Mon Aug 3 23:37:30 2026, metric_value: 2774.47
timestamp: Sun Aug 2 23:07:44 2026, metric_value: 2729.92
timestamp: Sat Aug 1 23:09:53 2026, metric_value: 2732.18
timestamp: Fri Jul 31 23:09:55 2026, metric_value: 2486.15
timestamp: Thu Jul 30 23:08:38 2026, metric_value: 2467.28
timestamp: Wed Jul 29 22:53:13 2026, metric_value: 2283.78
timestamp: Tue Jul 28 23:26:00 2026, metric_value: 2327.85
timestamp: Mon Jul 27 23:07:24 2026, metric_value: 2777.71
timestamp: Sun Jul 26 23:03:46 2026, metric_value: 2858.59 <---- Anomaly
timestamp: Sat Jul 25 22:54:07 2026, metric_value: 2398.88
timestamp: Fri Jul 24 23:00:38 2026, metric_value: 2379.31
timestamp: Thu Jul 23 23:14:49 2026, metric_value: 2403.32
timestamp: Wed Jul 22 22:56:31 2026, metric_value: 2223.12
timestamp: Tue Jul 21 22:56:42 2026, metric_value: 2191.53
timestamp: Mon Jul 20 22:55:29 2026, metric_value: 2081.13
timestamp: Sun Jul 19 22:49:13 2026, metric_value: 2286.14
timestamp: Sat Jul 18 22:48:21 2026, metric_value: 2334.37
timestamp: Fri Jul 17 22:44:25 2026, metric_value: 2140.38
timestamp: Thu Jul 16 22:48:39 2026, metric_value: 2199.59
```
Contributor guide
Research direction
Start with the Triage performance alert issues section in sdks/python/apache_beam/testing/analyzers/README.md, then inspect the linked test_cloudml_benchmark_test.py entry point around line 104. Compare the runtime history with the reported anomaly and follow the documented triage process. Done means the alert is classified as a real change or noise, with its cause and validation recorded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering, machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100