iree-org / iree-org/iree-comparative-benchmark
[Migration] Update dashboard to process results from openxla-benchmark
- Dominant language
- Python
- Stars
- 10
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
Benchmark results uploaded from openxla-benchmark are a little different from iree-samples.
Here is the example from T5_LARGE_FP32_JAX benchmarks:
From iree-samples: https://gist.github.com/pzread/2be3b2db7c0ffa14518085f08e33b815
From openxla-benchmarks: https://gist.github.com/pzread/cf3e089d0a4e0d5c452aebc1df6dbc44
The major differences are:
- Changes in benchmark id and benchmark name
- Changes in some metadata values (e.g. `data_type` and `device` field)
- Changes in the format of `python_environment` field
I think the pain point will be changing the benchmark id, which means we might need a backfill of the historical data. We can still show new data in the dashboard first then backfill the previous data
Contributor guide
Assessment
This issue has not been assessed yet.