Unify metrics output type
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 726
- Avg merge
- 5d 21h
- Merged PRs (30d)
- 5
Description
🚀 Feature
The idea is to verify the output type for all metrics (output of compute function) and update the docs accordingly.
In general, metric's output should be a float number. In some particular cases, like Recall/Precision with average=False, the output is a torch tensor. So, let's see and decide if the output of compute() method can be :
def compute() -> floatdef compute() -> Union[float, torch.Tensor]and tensor is on CPUdef compute() -> torch.Tensorwith tensor on CPU
To address this FR, we have to make sure for each metric what kind of type it supposes to return and update the docs accordingly.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inventorying each metric's compute() method and its current documentation. Check the output type for standard metrics and cases such as Recall or Precision with average=False, then decide on the supported contract. Done means every metric's output type is verified and the documentation reflects the chosen behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100