Kotlin / Kotlin/dataframe

Allow mixed number types for `Comparable`/selecting statistics

Open
#1,113 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Kotlin
Stars
1.1k
Forks
83
Avg merge
4d 12h
Merged PRs (30d)
30

Description

Related to https://github.com/Kotlin/dataframe/issues/961, https://github.com/Kotlin/dataframe/pull/1108.

After https://github.com/Kotlin/dataframe/pull/1108 is merged, `min`/`max` aggregations will only support operations on values that are self-comparable. This includes Dates, specific numbers, strings, etc.

Other aggregations, like `mean` and `sum` support calculating statistics with number unification, so calculating the sum of an `Int`-column, a `Double`-column, and a column containing both floats and integers is no problem whatsoever. The result will always be `Double`.

`min`/`max` currently throws an exception when this is done. This is (sort-of) inline with the Kotlin stdlib, as you can only calculate the min/max of a self-comparable iterable as well. However, from a user-perspective, when dealing with data of many types, it's obvious what `columnOf(1, 2.0, 3.0f).min()` would return (`1`!) and they might be surprised when it doesn't work. `describe()` actually has a workaround for this.

I previously thought it was impossible due to overload resolution ambiguity, however, it's possible to create 3 overloads for each function like:

1. `fun ?> DataColumn.min(): T & Any` for normal comparables

2. `fun DataColumn.min(): T & Any where T : Number?, T : Comparable` for normal numbers

3. `fun DataColumn.min(): T & Any` for mixed number types

We might also need two new aggregator handlers:
- an input handler that allows either self-comparables or numbers
- a selecting-like aggregation handler that functions like `aggregateBy` in the sense that it returns the item at `indexOfAggregationResultSingleSequence` by default, such that the original type is preserved while the aggregation result is decided by the unified numbers.

The same will hold for `median` and `percentile`. These functions will already need to be split into 1. and 2. overloads because they have different return types. Adding a type 3. overload for mixed number types will not be much more difficult.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the min/max aggregation behavior after pull request 1108 and the related issue 961. Trace the existing aggregation handlers and the workaround used by describe(). Done means mixed numeric columns support min and max with unified-number comparison while preserving the original item type, with the corresponding median and percentile overloads also covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
kotlin
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.