rust-lang / rust-lang/hashbrown
Switching to GxHash?
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3k
- Forks
- 358
- Avg merge
- 11h 57m
- Merged PRs (30d)
- 2
Description
Context
GxHash is a new non-cryptographic hashing algorithm that outperforms all counterparts (of the same class, fxhash is slightly faster for tiny inputs but at the cost of much worse distribution/avalanche/collision) for all input sizes on both ARM and X86.
GxHash uses AES instrinsics for efficient bit mixing (like ahash does) but makes more extensive use of SIMD operations and ILP. This enables gxhash to have a much smaller bytecode than ahash (it is much simpler) while being faster. GxHash hashes are stable across ARM/X86/X86+AVX2 and the algorithm passes all SMHasher tests (ahash does not, at least on my ARM MacBook).
Benchmark
I ran the benchmarks with gxhash on my ARM MacBook m1 pro and here are the results (using GxHash 2.2.4)
test lookup_std_highbits ... bench: 8,965 ns/iter (+/- 75)
test lookup_ahash_highbits ... bench: 4,079 ns/iter (+/- 79)
test lookup_gxhash_highbits ... bench: 2,976 ns/iter (+/- 78)
test lookup_std_random ... bench: 9,033 ns/iter (+/- 183)
test lookup_ahash_random ... bench: 4,245 ns/iter (+/- 57)
test lookup_gxhash_random ... bench: 3,043 ns/iter (+/- 22)
test lookup_std_serial ... bench: 8,949 ns/iter (+/- 207)
test lookup_fail_ahash_serial ... bench: 4,419 ns/iter (+/- 78)
test lookup_gxhash_serial ... bench: 2,669 ns/iter (+/- 29)
From these results there is about +25/30% performance on lookup from using gxhash compared to ahash. This is using gxhash 2.2.4, but gxhash 3.0.0 is almost ready and performs even better. It would be interesting also to see that we benchmark results look like on other platforms.
I am joining a PR to this issue so that you can test it.
Todo?
GxHash security properties haven't been assessed yet, but given what was said in this issue, I was thinking that a solution for even faster hashing would be welcome.
There are probably a few things to address before gxhash can be considered as a default hasher (portability, no std, ...?) but I would like to get your opinion on this idea first.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No repository files, tests, or entry points are named. Start by reviewing the proposed GxHash integration and comparing its portability, no-std support, security properties, and benchmark results across platforms; done requires a project decision on whether it is suitable as the default hasher.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100