File integrity check
- Dominant language
- Python
- Stars
- 117
- Forks
- 51
- Avg merge
- 9h 17m
- Merged PRs (30d)
- 5
Description
Maybe it would be useful if we could expose means of assessing the integrity of a file. I can think of the following scenarios:
1. check if a file is correctly uploaded to S3. For this, first calculate the file's checksum using a hash function (e.g., MD5, SHA256), then compare it with S3 generated checksum. If the two hash values match, then the file is correctly upload; otherwise retry uploading the file.
2. Get a file's checksum without download the file (e.g., see [this](https://stackoverflow.com/questions/26415923/boto-get-md5-s3-file)). A good scenario for this would checking if a local copy match S3 version, if not, then a collaborator has changed the file, hence update the local copy.
It seems boto is exposing MD5 checksum of an object, however, I guess the challenge would be to (1) ensue if/how other providers expose checksum without downloading a file/object; (2) are all the providers use common hash function for checksum?
Contributor guide
No contributing guide indexed for this repository
Research direction
No files, tests, or entry points are named. Start by reviewing the existing cloud provider interfaces and boto's S3 checksum behavior, then determine whether upload verification and remote checksum retrieval can share a cross-provider contract; done means the supported providers and hash semantics are explicitly defined.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100