aboutcode-org / aboutcode-org/scancode-toolkit

Collect all GenAI licenses for correct detection

Ouverte
#4,567 4 commentaires 0 réactions 1 personne assignée Assignée à @Godzilaa Voir sur GitHub
Langage dominant
Python
Étoiles
2.6k
Forks
791
Merge moyen
1 j 12 h
PR mergées (30 j)
5

Description

We need to ensure that LLMs have a clear license. This is messy out there with a lot of new, weird, proprietary licenses.

There is something that would be super useful wrt. AI: find all these licenses!

The process would be:
- [ ] Collect the metadata files from all Hugging Face GenAi repos
- [ ] Scan GenAi metadata files with scancode
- [ ] Review GenAi metadata file scan results for correctness
- [ ] Create new licenses, and new license rules used @ HuggingFace

When cloning all the repos @ Hugging Face, these should shallow clones, without Git LFS and without xet large files, and possibly partial clone to focus only on a few files: model cards and licenses, and related metadata files. See for ideas:
- https://github.com/aboutcode-org/federatedcode/issues/82

Running ScanCode license detection on that and review the results to find all the funky new AI-licenses and notices may be best done in ScanCode.io for efficiency, but the licenses are ultimately created in ScanCode toolkit

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.