aboutcode-org / aboutcode-org/scancode-toolkit
files opened many times during scanning
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 791
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 5
Description
### Description
When running scancode with `strace` I can see that every file is opened 4 or 5 times and also read as many times accordingly. A small (filtered) snippet of `strace` output:
```
...
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.c", O_RDONLY|O_NONBLOCK) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.c", O_RDONLY|O_NONBLOCK) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.c", O_RDONLY|O_CLOEXEC) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.c", O_RDONLY|O_CLOEXEC) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.c", O_RDONLY|O_CLOEXEC) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.h", O_RDONLY|O_NONBLOCK) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.h", O_RDONLY|O_NONBLOCK) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.h", O_RDONLY|O_CLOEXEC) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.h", O_RDONLY|O_CLOEXEC) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.h", O_RDONLY|O_CLOEXEC) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.yo", O_RDONLY|O_NONBLOCK) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.yo", O_RDONLY|O_NONBLOCK) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.yo", O_RDONLY|O_CLOEXEC) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.yo", O_RDONLY|O_CLOEXEC) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzip.yo", O_RDONLY|O_CLOEXEC) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzipall", O_RDONLY|O_NONBLOCK) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzipall", O_RDONLY|O_NONBLOCK) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzipall", O_RDONLY|O_CLOEXEC) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/rzipall", O_RDONLY|O_CLOEXEC) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/stream.c", O_RDONLY|O_NONBLOCK) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/stream.c", O_RDONLY|O_NONBLOCK) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/stream.c", O_RDONLY|O_CLOEXEC) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/stream.c", O_RDONLY|O_CLOEXEC) = 4
[pid 392633] openat(AT_FDCWD, "/tmp/rzip-2.1/stream.c", O_RDONLY|O_CLOEXEC) = 4
...
```
This seems a bit excessive. I haven't yet tried with really large files, but if these are also read multiple times (instead of data being shared between scans) it could get quite expensive.
Contributor guide
Assessment
This issue has not been assessed yet.