dotnet / dotnet/machinelearning

[DataFrame] Product, Min, Max Aggregations returning 0 on GroupBy

Open
#5,704 0 comments 1 reaction 0 assignees View on GitHub
area-DataFrame
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

The `Min` and `Product` aggregations on a DataFrame group return 0.

The corresponding computation methods appear to start with a zero value when selecting values based on the indices (as performed by `GroupBy`).

E.g.

```csharp
public void Min(PrimitiveColumnContainer column, IEnumerable rows, out byte ret)
{
ret = default;
```
Later the code compares `ret` with the current value and updates like so:
```csharp
ret = (byte)(Math.Min(readOnlySpan[(int)row], ret));
```

If all values are greater than zero, Min ends up returning 0.

In summary:
- `Min` returns 0 when all matching values are greater than zero.
- `Max` returns 0 when all matching values are less than zero.
- `Product` returns 0 with any matching values.

Example code showing the error:
```csharp
var foo = new PrimitiveDataFrameColumn(
"Foo",
Enumerable.Range(1, 10).SelectMany(_ => Enumerable.Range(1, 10))
);
var bar = new PrimitiveDataFrameColumn(
"Bar",
Enumerable.Range(1, 100)
);
var baz = new PrimitiveDataFrameColumn(
"Baz",
Enumerable.Range(-200, 100)
);

var df = new DataFrame(foo, bar);
var minFrame = df.GroupBy("Foo").Min("Bar");

// Here all the "Bar" values in minFrame are 0

var productFrame = df.GroupBy("Foo").Product("Bar");

// Here all the "Bar" values in productFrame are 0

var maxFrame = df.GroupBy("Foo").Max("Baz");

// Here all the "Baz" values in maxFrame are 0.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.