multiformats / multiformats/rust-multihash
No elegent way to stream hash given a hash code
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 166
- Forks
- 63
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
In previous multihash version, we used to be able to compute the digest in a streamed manner using MultihashDigest::input and it was possible to get a boxed MultihashDigest given a multihash.
I currently see no way of doing the same, which is an issue in some use cases.
For example, I need to validate a digest computed from a file. Since the file can be big, I want to use the new StatefulHasher trait. However, I found no way to get a trait object.
Here’s my code:
pub fn validate_file_checksum(expected_digest: &str, file_path: &Path) -> std::io::Result<bool> {
let (_, hash_data) = multibase::decode(expected_digest).map_err(|e| Error::new(ErrorKind::InvalidInput, e))?;
let expected_digest = Multihash::from_bytes(&hash_data).map_err(|e| Error::new(ErrorKind::InvalidInput, e))?;
let hash_code = multihash::Code::try_from(expected_digest.code()).map_err(|e| Error::new(ErrorKind::InvalidInput, e))?;
// FIXME: multihash new API is breaking this code for streaming hashing (checked for version 0.14)
//
//const BUF_SIZE: usize = 1024 * 128;
//let file = File::open(file_path)?;
//let mut reader = BufReader::with_capacity(BUF_SIZE, file);
//
//let hasher = todo!("get an appropriate trait object hasher given the hash code");
//
//loop {
// let length = {
// let buffer = reader.fill_buf()?;
// hasher.update(buffer);
// buffer.len()
// };
// if length == 0 {
// break;
// }
// reader.consume(length);
//}
//
//let digest_found = hasher.finalize();
//
// So instead, we read the whole file in memory:
let file_content = std::fs::read_to_string(file_path)?;
let digest_found = hash_code.digest(file_content.as_bytes());
Ok(expected_digest == digest_found)
}
If I overlooked something, please let me know!
Thank you
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the StatefulHasher trait, Code::try_from, and the hash_code.digest call shown in the issue. Trace how a hasher can be selected from a multihash code and how its finalized result is represented. Done means the validation example can process a large file incrementally and compare the resulting digest without reading the entire file into memory.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cryptography
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100