apache / apache/arrow-rs

Rank behaves differently from other systems

Open
#7,024 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

Something interesting about the rank function in arrow-rs is that it seems to behave differently compared to others.

Postgres:

> rank () → bigint
> Returns the rank of the current row, with gaps; that is, the row_number of the first row in its peer group.

- https://www.postgresql.org/docs/current/functions-window.html

DuckDB:

> Description The rank of the current row with gaps; same as row_number of its first peer.

- https://duckdb.org/docs/sql/functions/window_functions.html#rank

SQL Server:

> If two or more rows tie for a rank, each tied row receives the same rank. For example, if the two top salespeople have the same SalesYTD value, they are both ranked one.

- https://learn.microsoft.com/en-us/sql/t-sql/functions/rank-transact-sql?view=sql-server-ver16

It seems these other ones would have ties take the lower values, whereas for arrow-rs the existing rank function (and subsequently what is introduced here) takes the higher value; I guess this might be worth a broader discussion, as this PR itself currently follows the existing behaviour

:thinking:

_Originally posted by @Jefffrey in https://github.com/apache/arrow-rs/pull/6912#pullrequestreview-2561770071_

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing rank behavior in arrow-rs and the referenced discussion in pull request #6912. Before changing anything, establish the intended tie-ranking semantics against PostgreSQL, DuckDB, and SQL Server, then add or update tests that demonstrate the agreed behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.