rust-lang / rust-lang/hashbrown

Switching to GxHash?

Open
#487 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
3k
Forks
358
Avg merge
11h 57m
Merged PRs (30d)
2

Description

Context

GxHash is a new non-cryptographic hashing algorithm that outperforms all counterparts (of the same class, fxhash is slightly faster for tiny inputs but at the cost of much worse distribution/avalanche/collision) for all input sizes on both ARM and X86.

GxHash uses AES instrinsics for efficient bit mixing (like ahash does) but makes more extensive use of SIMD operations and ILP. This enables gxhash to have a much smaller bytecode than ahash (it is much simpler) while being faster. GxHash hashes are stable across ARM/X86/X86+AVX2 and the algorithm passes all SMHasher tests (ahash does not, at least on my ARM MacBook).

Benchmark

I ran the benchmarks with gxhash on my ARM MacBook m1 pro and here are the results (using GxHash 2.2.4)

test lookup_std_highbits          ... bench:       8,965 ns/iter (+/- 75)
test lookup_ahash_highbits        ... bench:       4,079 ns/iter (+/- 79)
test lookup_gxhash_highbits       ... bench:       2,976 ns/iter (+/- 78)

test lookup_std_random            ... bench:       9,033 ns/iter (+/- 183)
test lookup_ahash_random          ... bench:       4,245 ns/iter (+/- 57)
test lookup_gxhash_random         ... bench:       3,043 ns/iter (+/- 22)

test lookup_std_serial            ... bench:       8,949 ns/iter (+/- 207)
test lookup_fail_ahash_serial     ... bench:       4,419 ns/iter (+/- 78)
test lookup_gxhash_serial         ... bench:       2,669 ns/iter (+/- 29)

From these results there is about +25/30% performance on lookup from using gxhash compared to ahash. This is using gxhash 2.2.4, but gxhash 3.0.0 is almost ready and performs even better. It would be interesting also to see that we benchmark results look like on other platforms.

I am joining a PR to this issue so that you can test it.

Todo?

GxHash security properties haven't been assessed yet, but given what was said in this issue, I was thinking that a solution for even faster hashing would be welcome.

There are probably a few things to address before gxhash can be considered as a default hasher (portability, no std, ...?) but I would like to get your opinion on this idea first.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No repository files, tests, or entry points are named. Start by reviewing the proposed GxHash integration and comparing its portability, no-std support, security properties, and benchmark results across platforms; done requires a project decision on whether it is suitable as the default hasher.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.