aboutcode-org / aboutcode-org/scancode-toolkit

ScanCode process not terminating after having finished

未關閉
#4,134 3 則留言 1 個 reaction 已指派 0 人 在 GitHub 檢視
bug
主要語言
Python
星號
2.6k
分支
791
平均合併
1 天 12 小時
30 天內合併 PR
5

描述

### Description

We are scanning a large git repository that also has 3 submodules.
We execute the scan within a OpenShift pod, using

Python 3.11
ScanCode 32.3.1

The ScanCode scan runs as part of an ORT (OSS Review Toolkit) scan.
The following command line is used:

_scancode --copyright --license --info --strip-root --timeout 300 --max-in-memory 5000 --processes 4 --license-references --json /tmp/ort-ScanCode8749968816967782004/result.json /tmp/ort-DefaultProvenanceDownloader17726939893427938442_

As processes on the OpenShift pod, we see
* A java process (ORT worker)
* One Python process (the ScanCode CLI)
* 4 Python child processes

For some time, we see all the processes busy doing their work (using the tool _top_) , but at some point in time, the processes stop to work and are displayed in status _sleep_. This means, the scan job just hangs until eternity.

But on console output, we see actually that ScanCode has finished. It even cleaned up files:

````
$ find . -name scancode.stderr -exec cat {} \;
Setup plugins...
Collect file inventory...
Scan files for: info, licenses, copyrights with 4 process(es)...
Scanning done.
Summary: info, licenses, copyrights with 4 process(es)
Errors count: 0
Scan Speed: 7.35 files/sec. 104.36 KB/sec.
Initial counts: 15172 resource(s): 11782 file(s) and 3390 directorie(s)
Final counts: 15171 resource(s): 11782 file(s) and 3389 directorie(s) for 163.43 MB
Timings:
scan_start: 2025-02-06T092555.935638
scan_end: 2025-02-06T095310.228719
setup_scan:licenses: 4.85s
setup: 4.86s
inventory: 8.59s
scan:licenses: 5.89s
scan: 1603.61s
post-scan:license-references: 1.19s
post-scan: 1.19s
output:json: 4.94s
output: 4.94s
total: 1640.31s
Removing temporary files...done.
Setup plugins...
Collect file inventory...
Scan files for: info, licenses, copyrights with 4 process(es)...
````

So, for me (not being proficient in Python) it looks like the mother python process is finished and does a _ctx.exit(rc)_, but the process actually does not terminate. I guess it does not terminate, because the child processes are still alive.

Because the ScanCode mother process does not terminate, also the ORT scan step hangs.

### How To Reproduce

Sorry, the problem is reproduceable with an internal repository of ours, but I can give no simple way how to reproduce.

Nevertheless I decided to write this bug ticket, because others may run into the same situation.

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。