dotnet / dotnet/machinelearning-samples

Change restaurant dataset in Restaurant Violation Inspections sample

未关闭
#809 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
bug ModelBuilder
主要语言
PowerShell
星标
4.7k
派生
2.7k
平均合并
2 天 22 小时
30 天内合并 PR
1

描述

@luisquintanilla : We should replace the dataset used in [Tutorial: Classify the severity of restaurant health violations with Model Builder](https://github.com/dotnet/docs/blob/3803b263eae3f7575290a1da1142c34d1d4b5b7e/docs/machine-learning/tutorials/health-violation-classification-model-builder.md) in the docs repo, and the [Restaurant Violation Inspections](https://github.com/dotnet/machinelearning-samples/tree/9dc39d7f40c655029dae498e1d0de0406512f66d/samples/modelbuilder/MulticlassClassification_RestaurantViolations) sample in this samples repo.

See discussion: https://github.com/dotnet/docs/issues/17721

> Generally if you're getting 100% accuracy, you're either leaking, have too small of dataset to usefully measure the metrics, or the task is trivial.
>
> In this case the [dataset](https://github.com/luisquintanilla/machinelearning-samples/raw/AB1608219/samples/modelbuilder/MulticlassClassification_RestaurantViolations/RestaurantScores.zip) itself is leaking information or is trivial, depending on how you want to look at it. It has duplicates rows. The `violation_description` column perfectly predicts the label, `risk_category`, as it's the risk category of the given violation.
>
> There are 53,974 rows but only 363 are unique:
>
> ```shell
> $ wc -l RestaurantScores.tsv
> 53974
> $ sort RestaurantScores.tsv | uniq | wc -l
> 363
> ```
>
> See more information about leakage:
> https://en.wikipedia.org/wiki/Leakage_(machine_learning)
>
> # Next steps
> I would recommend replacing the dataset. Perhaps with the original dataset (with all columns).
>
> The dataset seems be [SF Restaurant Scores](https://data.sfgov.org/Health-and-Social-Services/Restaurant-Scores-LIVES-Standard/pyih-qa8i) ([download](Restaurant_Scores_-_LIVES_Standard.csv)). In that case, I would pose the problem either as regression or multi-class classification.
>
> Problem styles:
>
> * Regression -- predict `inspection_score` (**recommended**)
> * Multiclass classification -- predict `violation_description` while ignoring `risk_category` (or the other way around)
>
> For all tasks, I would split the dataset in to train/validate/test based on date (`inspection_date` column). With the oldest data in the train split, newer in validate, and most recent in test.
>
> Why split on time?
> This is to avoid leaking data between the dataset splits as this dataset is time-dependent, as the newer information better predicts the other newer rows than does the older rows. For example the `inspection_score` of a restaurant in June is likely closer to the July score than the January score.
>
> _... (preview of dataset [followed](https://github.com/dotnet/docs/issues/17721))..._

Issue also filed in docs repo to track that change -- https://github.com/dotnet/docs/issues/17962

贡献指南

打开贡献指南

调研方向

首先阅读 dotnet/docs issues 17721 和 17962 中链接的讨论,然后检查 docs/machine-learning/tutorials/health-violation-classification-model-builder.md 中的教程以及 samples/modelbuilder/MulticlassClassification_RestaurantViolations 下的示例。替换造成泄漏的 dataset,并根据所选任务调整示例和教程,按照讨论使用基于时间的训练、验证和测试划分;完成的标准是两个 repository 都一致地使用新的 dataset。

由索引模型根据 Issue 内容生成。

评估

技术栈
machine-learning
领域
documentation, machine-learning
Issue 类型
功能
难度
5/5
预计耗时
一周以上
活跃度
停滞
描述清晰度
基本清楚
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。