How easy it is to externalize the visibility of the benchmarks we already run continuously?
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 311
- Forks
- 139
- Avg merge
- 1h 28m
- Merged PRs (30d)
- 1
Description
This a pinpoint to not lost the benchmarking subthread started at https://github.com/tensorflow/tensorflow/pull/33945#issuecomment-652471742
Quoting @zongweiz
We have [TF2 benchmarks] (https://github.com/tensorflow/models/tree/master/official/benchmark) from every official model garden model and they continuously running internally, using PerfZero framework. We should be able to select a set of benchmarks and hook it up with github CI (jenkins). Added @sganeshb to this thread for information and we will follow up.
Another thought is to start CI tests using a selected set of tf_benchmarks
/cc @alextp @naveenjha123 @sganeshb
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked TensorFlow PR comment and the TF2 benchmarks under the official model benchmark directory. Review how PerfZero runs the benchmarks internally and clarify which benchmarks, CI system, and integration path are intended; the issue does not define a concrete completion condition.
Written by the indexing model from the issue text.
Assessment
- Domain
- build-system, ci-cd
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100