aboutcode-org / aboutcode-org/scancode-toolkit

Disk cache usage is slow

オープン
#1,014 コメント 3 件 リアクション 0 件 担当者 0 名 GitHub で見る
core and api documentation performance Priority: medium
主要言語
Python
スター
2.6k
フォーク
791
平均マージ
1日 12時間
マージ済み PR(30日)
5

説明

Running a file info scan on the source of Linux 4.15.7 takes a long time when the disk cache is being used.

Disk cache used (default `--max-in-memory 10000`):

```
$ scancode -i -n4 ~/Desktop/linux-stable-4.15.7 --json ~/Desktop/kernel.json
Setup plugins...
Collect file inventory...
Scan files for: info with 4 process(es)...
[####################] 62061
Scanning done.
Summary: info with 4 process(es)
Errors count: 0
Scan Speed: 65.67 files/sec. 461.51 KB/sec.
Initial counts: 12452 resource(s): 11039 file(s) and 1413 directorie(s)
Final counts: 12452 resource(s): 11039 file(s) and 1413 directorie(s) for 75.76 MB
Timings:
inventory: 1382.95s
scan: 168.11s
output:json: 30.49s
output: 30.49s
total: 1683.44s
```

For comparison, these are the results of having everything in memory (`--max-in-memory 0`):

```
$ scancode -i -n4 ~/Desktop/linux-stable-4.15.7 --json ~/Desktop/kernel.json --max-in-memory 0
Setup plugins...
Collect file inventory...
Scan files for: info with 4 process(es)...
[####################] 62061
Scanning done.
Summary: info with 4 process(es)
Errors count: 0
Scan Speed: 484.66 files/sec. 5.84 MB/sec.
Initial counts: 66438 resource(s): 62061 file(s) and 4377 directorie(s)
Final counts: 66438 resource(s): 62061 file(s) and 4377 directorie(s) for 747.61 MB
Timings:
inventory: 8.80s
scan: 128.05s
output:json: 9.31s
output: 9.31s
total: 147.48s
```

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。