aboutcode-org / aboutcode-org/scancode-toolkit

Collect all GenAI licenses for correct detection

Aperta
#4,567 4 commenti 0 reazioni 1 assegnatario Assegnata a @Godzilaa Vedi su GitHub
Lingua principale
Python
Stelle
2.6k
Fork
791
Merge medio
1g 12h
PR unite (30g)
5

Descrizione

We need to ensure that LLMs have a clear license. This is messy out there with a lot of new, weird, proprietary licenses.

There is something that would be super useful wrt. AI: find all these licenses!

The process would be:
- [ ] Collect the metadata files from all Hugging Face GenAi repos
- [ ] Scan GenAi metadata files with scancode
- [ ] Review GenAi metadata file scan results for correctness
- [ ] Create new licenses, and new license rules used @ HuggingFace

When cloning all the repos @ Hugging Face, these should shallow clones, without Git LFS and without xet large files, and possibly partial clone to focus only on a few files: model cards and licenses, and related metadata files. See for ideas:
- https://github.com/aboutcode-org/federatedcode/issues/82

Running ScanCode license detection on that and review the results to find all the funky new AI-licenses and notices may be best done in ScanCode.io for efficiency, but the licenses are ultimately created in ScanCode toolkit

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.