aboutcode-org / aboutcode-org/scancode-toolkit

Feature Request: Parallalize all steps of scancode if possible

Abierto
#2,725 3 comentarios 0 reacciones 0 asignados Ver en GitHub
new feature
Lenguaje dominante
Python
Estrellas
2.6k
Forks
791
Merge medio
1 d 12 h
PR fusionados (30 d)
5

Descripción

## Short Description
In all but the file scanning step, scancode only uses one core, resulting in a low CPU load and a long runtime.

## Select Category
- [X] Enhancement

## **How This Feature will help you/your organization**
It should reduce the overall runtime substantially.

## **Possible Solution/Implementation Details**
I started scancode on a large codebase with parameter _-n 30_. After two hours (on a 32 core CPU), it was still in the state _Collect file inventory..._. The entire code base is on a M.2 SSD. The task manager shows little CPU, Memory, and IO activity.

Please parallelize all steps of scancode, including the file collection step.

## **Hardware / Software used**
AMD Ryzen 9 3950X (16 core, 32 thread), 32GB RAM, M.2 SSD 970 EVO Plus 1TB
Windows 10 Enterprise LTSC (1809), Python 3.9, Scancode 30.0.0

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.