aboutcode-org / aboutcode-org/scancode-toolkit
Collect all GenAI licenses for correct detection
- Lenguaje dominante
- Python
- Estrellas
- 2.6k
- Forks
- 791
- Merge medio
- 1 d 12 h
- PR fusionados (30 d)
- 5
Descripción
We need to ensure that LLMs have a clear license. This is messy out there with a lot of new, weird, proprietary licenses.
There is something that would be super useful wrt. AI: find all these licenses!
The process would be:
- [ ] Collect the metadata files from all Hugging Face GenAi repos
- [ ] Scan GenAi metadata files with scancode
- [ ] Review GenAi metadata file scan results for correctness
- [ ] Create new licenses, and new license rules used @ HuggingFace
When cloning all the repos @ Hugging Face, these should shallow clones, without Git LFS and without xet large files, and possibly partial clone to focus only on a few files: model cards and licenses, and related metadata files. See for ideas:
- https://github.com/aboutcode-org/federatedcode/issues/82
Running ScanCode license detection on that and review the results to find all the funky new AI-licenses and notices may be best done in ScanCode.io for efficiency, but the licenses are ultimately created in ScanCode toolkit
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.