LAION-AI / LAION-AI/CLIP_benchmark
add fairness related evaluations
Open
Nobody has claimed this yet.
new datasets
- Dominant language
- Python
- Stars
- 814
- Forks
- 103
- PR merge metrics
- No merged PRs in 30d
Description
could be good to have some fairness related datasets, e.g., from https://arxiv.org/abs/2108.02818. curious how LAION CLIP compares to OAI CLIP.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the fairness datasets and evaluation approach described in the linked paper, then inspect the repository's existing evaluation workflow. Define which datasets and metrics should be added and how LAION CLIP should be compared with OAI CLIP; done means the fairness evaluations are reproducible and the comparison results are reported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100