dotnet / dotnet/machinelearning

[AutoML] Reservoir sample dataset statistics

Open
#3,778 0 comments 0 reactions 0 assignees View on GitHub
area-AutoML enhancement Priority:2
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

Currently dataset statistics within AutoML are calculated from the first 1,000 rows of a dataset. Instead, we should be calculating statistics from a random sample of 1,000 rows. (First 1,000 rows could be biased if they are sorted by label, any other column, time of collection, etc.) We can use reservoir sampling to obtain a random sample of a fixed size in a single pass over the dataset

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.