aboutcode-org / aboutcode-org/scancode-toolkit
ScanCode process not terminating after having finished
- Vorherrschende Sprache
- Python
- Sterne
- 2.6k
- Forks
- 791
- Ø Merge
- 1 T. 12 Std.
- Gemergte PRs (30 T.)
- 5
Beschreibung
### Description
We are scanning a large git repository that also has 3 submodules.
We execute the scan within a OpenShift pod, using
Python 3.11
ScanCode 32.3.1
The ScanCode scan runs as part of an ORT (OSS Review Toolkit) scan.
The following command line is used:
_scancode --copyright --license --info --strip-root --timeout 300 --max-in-memory 5000 --processes 4 --license-references --json /tmp/ort-ScanCode8749968816967782004/result.json /tmp/ort-DefaultProvenanceDownloader17726939893427938442_
As processes on the OpenShift pod, we see
* A java process (ORT worker)
* One Python process (the ScanCode CLI)
* 4 Python child processes
For some time, we see all the processes busy doing their work (using the tool _top_) , but at some point in time, the processes stop to work and are displayed in status _sleep_. This means, the scan job just hangs until eternity.
But on console output, we see actually that ScanCode has finished. It even cleaned up files:
````
$ find . -name scancode.stderr -exec cat {} \;
Setup plugins...
Collect file inventory...
Scan files for: info, licenses, copyrights with 4 process(es)...
Scanning done.
Summary: info, licenses, copyrights with 4 process(es)
Errors count: 0
Scan Speed: 7.35 files/sec. 104.36 KB/sec.
Initial counts: 15172 resource(s): 11782 file(s) and 3390 directorie(s)
Final counts: 15171 resource(s): 11782 file(s) and 3389 directorie(s) for 163.43 MB
Timings:
scan_start: 2025-02-06T092555.935638
scan_end: 2025-02-06T095310.228719
setup_scan:licenses: 4.85s
setup: 4.86s
inventory: 8.59s
scan:licenses: 5.89s
scan: 1603.61s
post-scan:license-references: 1.19s
post-scan: 1.19s
output:json: 4.94s
output: 4.94s
total: 1640.31s
Removing temporary files...done.
Setup plugins...
Collect file inventory...
Scan files for: info, licenses, copyrights with 4 process(es)...
````
So, for me (not being proficient in Python) it looks like the mother python process is finished and does a _ctx.exit(rc)_, but the process actually does not terminate. I guess it does not terminate, because the child processes are still alive.
Because the ScanCode mother process does not terminate, also the ORT scan step hangs.
### How To Reproduce
Sorry, the problem is reproduceable with an internal repository of ours, but I can give no simple way how to reproduce.
Nevertheless I decided to write this bug ticket, because others may run into the same situation.
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.