dovudo / dovudo/DwScript

Implemet hashing

Open
#20 0 comments 0 reactions 0 assignees View on GitHub
feature good first issue help wanted
Dominant language
Python
Stars
9
Forks
6
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**

I'll fix a problem of duplicate files.
This solution is simple and good for first issue.

**Describe the solution you'd like**

Hash files by "xxhash" algorithm, because it's pretty fast algorithm.

- Make hashing each file.
- Add hash to store or txt file in main directory.
- Check hash of every downloaded file with hash store.
- Skip same files.

**Additional context**
Xxhash benchmark
![Screenshot from 2020-05-04 16-51-01](https://user-images.githubusercontent.com/12629827/80973197-b8971800-8e27-11ea-887c-4b9d0f1da06e.png)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.