Investigate possibility of system metrics usage in portable performance tests
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
We currently use [TimeMonitor.java](https://github.com/apache/beam/blob/master/sdks/java/testing/test-utils/src/main/java/org/apache/beam/sdk/testutils/metrics/TimeMonitor.java) and [MeasureTime.py](https://github.com/apache/beam/blob/1988284a89b10b60eea48325f8a3b370b551c77c/sdks/python/apache_beam/testing/load_tests/load_test_metrics_utils.py#L406) DoFns to collect runtime in both portable and non-portable performance tests. However, in portable tests it seems to be possible to use [TOTAL_TIME_MSECS ](https://github.com/apache/beam/blob/1988284a89b10b60eea48325f8a3b370b551c77c/model/pipeline/src/main/proto/metrics.proto#L130)for collecting execution time. Other system metrics are available as well (size, bundle size etc).
It seems like a good way to simplify things and get more useful metrics from portable jobs so it is worth investigating ways of using it in performance tests.
Imported from Jira [BEAM-8826](https://issues.apache.org/jira/browse/BEAM-8826). Original Jira may contain additional context.
Reported by: ŁukaszG.
Contributor guide
Assessment
This issue has not been assessed yet.