aboutcode-org / aboutcode-org/scancode-toolkit

Feature Request: Parallalize all steps of scancode if possible

Aberta
#2,725 3 comentários 0 reações 0 responsáveis Ver no GitHub
new feature
Linguagem predominante
Python
Estrelas
2.6k
Forks
791
Merge médio
1d 12h
PRs com merge (30d)
5

Descrição

## Short Description
In all but the file scanning step, scancode only uses one core, resulting in a low CPU load and a long runtime.

## Select Category
- [X] Enhancement

## **How This Feature will help you/your organization**
It should reduce the overall runtime substantially.

## **Possible Solution/Implementation Details**
I started scancode on a large codebase with parameter _-n 30_. After two hours (on a 32 core CPU), it was still in the state _Collect file inventory..._. The entire code base is on a M.2 SSD. The task manager shows little CPU, Memory, and IO activity.

Please parallelize all steps of scancode, including the file collection step.

## **Hardware / Software used**
AMD Ryzen 9 3950X (16 core, 32 thread), 32GB RAM, M.2 SSD 970 EVO Plus 1TB
Windows 10 Enterprise LTSC (1809), Python 3.9, Scancode 30.0.0

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.