huggingface / huggingface/model-evaluator
Add FLOPS to reporting of results
Open
- Dominant language
- Python
- Stars
- 14
- Forks
- 8
- Avg merge
- 34m
- Merged PRs (30d)
- 1
Description
Per @lewtun:
```
Should we also report FLOPs (or similar)? I believe one can get this from the Trainer pretty easily (if not by default): https://huggingface.co/docs/transformers/v4.22.2/en/main_classes/trainer#transformers.Trainer.floating_point_ops
This would allow people to plot performance vs compute, which is arguably more meaningful (modulo caveats around hardware differences)
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.