Implement xxhash algorithms as part of the expression API
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
I am currently in need of using datafusion SQL query engine (through delta-rs merge operation) to hash with a specific hashing algorithm.
https://xxhash.com/
https://github.com/Cyan4973/xxHash
Rust implementation : https://crates.io/crates/twox-hash
### Describe the solution you'd like
I'd like the hashing functions xxhash32 and xxhash64 to be part of [datafusion expression API](https://datafusion.apache.org/user-guide/expressions.html) so the functions can be used in SQL context for hashing data.
Contributor guide
Research direction
Start with the linked DataFusion expression API documentation and the twox-hash crate to understand the requested xxhash32 and xxhash64 functions. Identify the relevant expression and SQL integration tests; the work is done when both functions can be called from SQL and their hash results are verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100