awslabs / awslabs/aws-cv-task2vec

task2vec complexity using normalized/standardized values

Open
#16 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
126
Forks
31
PR merge metrics
No merged PRs in 30d

Description

I saw the task2vec complexity using l1. But the l1 complexities might not be comparable across data sets. How would you suggest to adjust this metric s.t. its directly comparable accross data sets?

My suggestion is to divide perhaps by the std l1 complexity for that data set e.g.:

```
standardized_complexity = avg_complexity(task2vecs_list, l1) / unbiased_std_complexity(task2vecs_list, l1)
```

thoughts? This does assume normality and n>=30. Histograms showing normality might be useful.

----

related: https://stats.stackexchange.com/questions/604296/how-does-one-create-comparable-metrics-when-the-original-metrics-are-not-compara

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.