aboutcode-org / aboutcode-org/scancode-toolkit
Feature Request: Parallalize all steps of scancode if possible
- Lingua principale
- Python
- Stelle
- 2.6k
- Fork
- 791
- Merge medio
- 1g 12h
- PR unite (30g)
- 5
Descrizione
## Short Description
In all but the file scanning step, scancode only uses one core, resulting in a low CPU load and a long runtime.
## Select Category
- [X] Enhancement
## **How This Feature will help you/your organization**
It should reduce the overall runtime substantially.
## **Possible Solution/Implementation Details**
I started scancode on a large codebase with parameter _-n 30_. After two hours (on a 32 core CPU), it was still in the state _Collect file inventory..._. The entire code base is on a M.2 SSD. The task manager shows little CPU, Memory, and IO activity.
Please parallelize all steps of scancode, including the file collection step.
## **Hardware / Software used**
AMD Ryzen 9 3950X (16 core, 32 thread), 32GB RAM, M.2 SSD 970 EVO Plus 1TB
Windows 10 Enterprise LTSC (1809), Python 3.9, Scancode 30.0.0
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.