aboutcode-org / aboutcode-org/scancode-toolkit

Hash is unexpectedly None for empty files

Open
#4,428 11 comments 0 reactions 0 assignees View on GitHub
documentation
Dominant language
Python
Stars
2.6k
Forks
791
Avg merge
1d 12h
Merged PRs (30d)
5

Description

Generating hashes for empty files will always return `None`, which is not documented and different from the usual hashing algorithms as well as contradicting the SPDX standard.

Example:

```python3
from commoncode.hash import sha1
from hashlib import sha1 as sha1_hashlib
from tempfile import NamedTemporaryFile

with NamedTemporaryFile() as temporary_file:
temporary_file.write(b'')
temporary_file.seek(0)
print(sha1(location=temporary_file.name))
print(sha1_hashlib(string=temporary_file.read(), used_for_security=False).hexdigest())
```

The reason seems to be that https://github.com/aboutcode-org/commoncode/blob/878be6140deac30e2b95fb0fad9eb8feca015fc8/src/commoncode/hash.py#L38 does not use `msg is not None`, but basically `bool(msg)`, which is `False` for empty inputs as well.

Replacing the line with

```python3
self.h = msg is not None and hmodule(msg).digest()[:self.digest_size] or None
```

(as well as replacing the same pattern in `sha1_git_hasher`) seems to fix this issue.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.