aboutcode-org / aboutcode-org/scancode-toolkit

Scan freezes when scanning big repositories (dotnet/corefx or dotnet/runtime)

Ouverte
#4,508 8 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
bug
Langage dominant
Python
Étoiles
2.6k
Forks
791
Merge moyen
1 j 12 h
PR mergées (30 j)
5

Description

### Description

I'm running ScanCode v32.4.0 under Windows with Python v3.10. We're running ScanCode for .NET OSS dependencies using OSS Review Toolkit (ORT). We have some dependencies to libraries from Microsoft, stored in dotnet/runtime and dotnet/corefx GitHub repos. Both repositories are really big. Unfortunately, even if ORT configured to use the certain path inside of this repository (e.g. src/System.Threading.Tasks.Extensions), ORT starts a scan of the whole repository. However, running ORT on my PC for the whole repository results in freezing of ScanCode after certain time (20 minutes). E.g. running for commit: https://github.com/dotnet/corefx/tree/7601f4f6225089ffb291dc7d58293c7bbf5c5d4f/
Windows doesn't detect the python.exe as hanging or frozen, howeve raccording to console outputs, no progress happens even after many hours or days. If I scan the subfolders of big repository separately, the problem doesn't happen.

### How To Reproduce

Checkout commit "https://github.com/dotnet/corefx/tree/7601f4f6225089ffb291dc7d58293c7bbf5c5d4f/"
Run ScanCode:
`scancode.bat --info --copyright --license --verbose --strip-root --timeout 5 --ignore "*.sln" --ignore "*.csproj" --ignore "*.htm" --ignore "*.html" --ignore "*.pdf" --processes 6 --license-references --spdx-tv D:\Temp\result.yml "{PATH_TO_REVISION_CHECKOUT}"`

After about 10-30 minutes there will be no progress for scan code.

### System configuration

* What OS are you running on? Windows
* What version of scancode-toolkit was used to generate the scan file? v32.4.0
* What installation method was used to install/run scancode? (pip/source download/other): binary download

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.