Clarify `rehash` setting in hash utils
- Lingua principale
- Rust
- Stelle
- 9.3k
- Fork
- 2.4k
- Merge medio
- 3g 11h
- PR unite (30g)
- 360
Descrizione
### Is your feature request related to a problem or challenge?
In our hash util functions, we have a `rehash` argument across many of them:
https://github.com/apache/datafusion/blob/b80bf2ca8ef74900fee96a1cc169bdedf53b36fc/datafusion/common/src/hash_utils.rs#L185-L190
https://github.com/apache/datafusion/blob/b80bf2ca8ef74900fee96a1cc169bdedf53b36fc/datafusion/common/src/hash_utils.rs#L230-L235
https://github.com/apache/datafusion/blob/b80bf2ca8ef74900fee96a1cc169bdedf53b36fc/datafusion/common/src/hash_utils.rs#L282-L287
It's not clearly obvious why we do this from the code alone; it seems it used to be named `multi_col` and would be true if we needed to hash multiple columns, but was changed in #6816 to also skip rehash if it is the first column, for performance reasons.
- It seems dictionary function also still calls it `multi_col`
I also found it confusing how certain hash functions don't have a rehash parameter; specifically the nested types such as list, struct, etc.
https://github.com/apache/datafusion/blob/b80bf2ca8ef74900fee96a1cc169bdedf53b36fc/datafusion/common/src/hash_utils.rs#L447-L451
https://github.com/apache/datafusion/blob/b80bf2ca8ef74900fee96a1cc169bdedf53b36fc/datafusion/common/src/hash_utils.rs#L475-L479
https://github.com/apache/datafusion/blob/b80bf2ca8ef74900fee96a1cc169bdedf53b36fc/datafusion/common/src/hash_utils.rs#L510-L515
### Describe the solution you'd like
Add some documentation explaining why we have a `rehash` parameter across the functions. Also look into adding `rehash` parameter for those hash functions missing them. If this parameter was omitted on purpose for such functions, leave an explanation of why this is the case.
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Inizia in datafusion/common/src/hash_utils.rs con le funzioni di hash intorno alle righe 185-190, 230-235 e 282-287, quindi esamina le funzioni per dizionari e tipi annidati intorno alle righe 447-451, 475-479 e 510-515. Controlla la cronologia di #6816 per comprendere il comportamento di rehash. Il lavoro è completato quando il parametro rehash è documentato in modo coerente e i parametri mancanti vengono aggiunti oppure viene spiegata la loro omissione intenzionale.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- rust
- Ambito
- backend
- Tipo di issue
- Funzionalità
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 45/100