aboutcode-org / aboutcode-org/scancode-toolkit

Scan freezes when scanning big repositories (dotnet/corefx or dotnet/runtime)

Abierto
#4,508 8 comentarios 0 reacciones 0 asignados Ver en GitHub
bug
Lenguaje dominante
Python
Estrellas
2.6k
Forks
791
Merge medio
1 d 12 h
PR fusionados (30 d)
5

Descripción

### Description

I'm running ScanCode v32.4.0 under Windows with Python v3.10. We're running ScanCode for .NET OSS dependencies using OSS Review Toolkit (ORT). We have some dependencies to libraries from Microsoft, stored in dotnet/runtime and dotnet/corefx GitHub repos. Both repositories are really big. Unfortunately, even if ORT configured to use the certain path inside of this repository (e.g. src/System.Threading.Tasks.Extensions), ORT starts a scan of the whole repository. However, running ORT on my PC for the whole repository results in freezing of ScanCode after certain time (20 minutes). E.g. running for commit: https://github.com/dotnet/corefx/tree/7601f4f6225089ffb291dc7d58293c7bbf5c5d4f/
Windows doesn't detect the python.exe as hanging or frozen, howeve raccording to console outputs, no progress happens even after many hours or days. If I scan the subfolders of big repository separately, the problem doesn't happen.

### How To Reproduce

Checkout commit "https://github.com/dotnet/corefx/tree/7601f4f6225089ffb291dc7d58293c7bbf5c5d4f/"
Run ScanCode:
`scancode.bat --info --copyright --license --verbose --strip-root --timeout 5 --ignore "*.sln" --ignore "*.csproj" --ignore "*.htm" --ignore "*.html" --ignore "*.pdf" --processes 6 --license-references --spdx-tv D:\Temp\result.yml "{PATH_TO_REVISION_CHECKOUT}"`

After about 10-30 minutes there will be no progress for scan code.

### System configuration

* What OS are you running on? Windows
* What version of scancode-toolkit was used to generate the scan file? v32.4.0
* What installation method was used to install/run scancode? (pip/source download/other): binary download

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.