BartMassey / BartMassey/simhash
Could you please clarify how the algorithm works?
- Dominant language
- C
- Stars
- 12
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
The manpage says:
> The algorithm used by simhash is Manassas' "shingleprinting" algorithm (see BIBLIOGRAPHY below): take a hash of every m-byte subsequence of the file, and retain the n of these hashes that are numerically smallest
Unfortunately, the url in the BIBLIOGRAPHY section (http://research.microsoft.com/research/sv/PageTurner/similarity.htm) is no longer reachable.
What I understand: You hash every subsequence of m bytes (with m=8 by default), and you keep a count of how many time each hash appears. Then you select n hashes (by default n=128) which will be your output. My question is: how do you select those n features?
What do you mean exactly when you say you retain the n "numerically smallest"? Do you select the n hashes that have appeared the least? Or do you select the n hashes that have the smallest value when interpreted as numbers?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the simhash manpage passage and its BIBLIOGRAPHY section, then compare the wording with the implementation if needed. Done means the documentation explicitly defines “numerically smallest,” explains how the n features are selected, and addresses the unavailable reference URL.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 38/100