google / google/xls

Move functionality out of benchmark_main

Open
#835 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
1.9k
Forks
283
Avg merge
2d 10h
Merged PRs (30d)
135

Description

We should reduce the functionality of `benchmark_main`. It basically tries to replicate various stages of the build process to gather metrics. I think a less error prone process would be to gather metrics from build artifacts (schedule, block IR (which we don't yet dump), etc) and not actually run anything in benchmark main. This would avoid skew between build targets and what benchmark_main actually does. There is the issue of how to measure the time it takes to build the artifacts (optimization time, etc) but perhaps that might be solved by indicating how the build artifacts were generated perhaps as comments at the top of the artifact files, or maybe a separate artifact file.

One approach might be for the build rules to dump `.metrics` files with data for benchmark script to scrap. I guess the metrics file could also include the instructions on how to rebuild the artifact in order to time how long it takes.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.