huggingface / huggingface/model-evaluator

Add FLOPS to reporting of results

Open
#66 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
14
Forks
8
Avg merge
34m
Merged PRs (30d)
1

Description

Per @lewtun:

```
Should we also report FLOPs (or similar)? I believe one can get this from the Trainer pretty easily (if not by default): https://huggingface.co/docs/transformers/v4.22.2/en/main_classes/trainer#transformers.Trainer.floating_point_ops

This would allow people to plot performance vs compute, which is arguably more meaningful (modulo caveats around hardware differences)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.