dotnet / dotnet/machinelearning-samples

Change restaurant dataset in Restaurant Violation Inspections sample

Abierto
#809 0 comentarios 0 reacciones 0 asignados Ver en GitHub
bug ModelBuilder
Lenguaje dominante
PowerShell
Estrellas
4.7k
Forks
2.7k
Merge medio
2 d 22 h
PR fusionados (30 d)
1

Descripción

@luisquintanilla : We should replace the dataset used in [Tutorial: Classify the severity of restaurant health violations with Model Builder](https://github.com/dotnet/docs/blob/3803b263eae3f7575290a1da1142c34d1d4b5b7e/docs/machine-learning/tutorials/health-violation-classification-model-builder.md) in the docs repo, and the [Restaurant Violation Inspections](https://github.com/dotnet/machinelearning-samples/tree/9dc39d7f40c655029dae498e1d0de0406512f66d/samples/modelbuilder/MulticlassClassification_RestaurantViolations) sample in this samples repo.

See discussion: https://github.com/dotnet/docs/issues/17721

> Generally if you're getting 100% accuracy, you're either leaking, have too small of dataset to usefully measure the metrics, or the task is trivial.
>
> In this case the [dataset](https://github.com/luisquintanilla/machinelearning-samples/raw/AB1608219/samples/modelbuilder/MulticlassClassification_RestaurantViolations/RestaurantScores.zip) itself is leaking information or is trivial, depending on how you want to look at it. It has duplicates rows. The `violation_description` column perfectly predicts the label, `risk_category`, as it's the risk category of the given violation.
>
> There are 53,974 rows but only 363 are unique:
>
> ```shell
> $ wc -l RestaurantScores.tsv
> 53974
> $ sort RestaurantScores.tsv | uniq | wc -l
> 363
> ```
>
> See more information about leakage:
> https://en.wikipedia.org/wiki/Leakage_(machine_learning)
>
> # Next steps
> I would recommend replacing the dataset. Perhaps with the original dataset (with all columns).
>
> The dataset seems be [SF Restaurant Scores](https://data.sfgov.org/Health-and-Social-Services/Restaurant-Scores-LIVES-Standard/pyih-qa8i) ([download](Restaurant_Scores_-_LIVES_Standard.csv)). In that case, I would pose the problem either as regression or multi-class classification.
>
> Problem styles:
>
> * Regression -- predict `inspection_score` (**recommended**)
> * Multiclass classification -- predict `violation_description` while ignoring `risk_category` (or the other way around)
>
> For all tasks, I would split the dataset in to train/validate/test based on date (`inspection_date` column). With the oldest data in the train split, newer in validate, and most recent in test.
>
> Why split on time?
> This is to avoid leaking data between the dataset splits as this dataset is time-dependent, as the newer information better predicts the other newer rows than does the older rows. For example the `inspection_score` of a restaurant in June is likely closer to the July score than the January score.
>
> _... (preview of dataset [followed](https://github.com/dotnet/docs/issues/17721))..._

Issue also filed in docs repo to track that change -- https://github.com/dotnet/docs/issues/17962

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.