dotnet / dotnet/machinelearning-samples
Change restaurant dataset in Restaurant Violation Inspections sample
- Ngôn ngữ chính
- PowerShell
- Star
- 4.7k
- Fork
- 2.7k
- Merge trung bình
- 2 ngày 22 giờ
- Pull request đã merge (30 ngày)
- 1
Mô tả
@luisquintanilla : We should replace the dataset used in [Tutorial: Classify the severity of restaurant health violations with Model Builder](https://github.com/dotnet/docs/blob/3803b263eae3f7575290a1da1142c34d1d4b5b7e/docs/machine-learning/tutorials/health-violation-classification-model-builder.md) in the docs repo, and the [Restaurant Violation Inspections](https://github.com/dotnet/machinelearning-samples/tree/9dc39d7f40c655029dae498e1d0de0406512f66d/samples/modelbuilder/MulticlassClassification_RestaurantViolations) sample in this samples repo.
See discussion: https://github.com/dotnet/docs/issues/17721
> Generally if you're getting 100% accuracy, you're either leaking, have too small of dataset to usefully measure the metrics, or the task is trivial.
>
> In this case the [dataset](https://github.com/luisquintanilla/machinelearning-samples/raw/AB1608219/samples/modelbuilder/MulticlassClassification_RestaurantViolations/RestaurantScores.zip) itself is leaking information or is trivial, depending on how you want to look at it. It has duplicates rows. The `violation_description` column perfectly predicts the label, `risk_category`, as it's the risk category of the given violation.
>
> There are 53,974 rows but only 363 are unique:
>
> ```shell
> $ wc -l RestaurantScores.tsv
> 53974
> $ sort RestaurantScores.tsv | uniq | wc -l
> 363
> ```
>
> See more information about leakage:
> https://en.wikipedia.org/wiki/Leakage_(machine_learning)
>
> # Next steps
> I would recommend replacing the dataset. Perhaps with the original dataset (with all columns).
>
> The dataset seems be [SF Restaurant Scores](https://data.sfgov.org/Health-and-Social-Services/Restaurant-Scores-LIVES-Standard/pyih-qa8i) ([download](Restaurant_Scores_-_LIVES_Standard.csv)). In that case, I would pose the problem either as regression or multi-class classification.
>
> Problem styles:
>
> * Regression -- predict `inspection_score` (**recommended**)
> * Multiclass classification -- predict `violation_description` while ignoring `risk_category` (or the other way around)
>
> For all tasks, I would split the dataset in to train/validate/test based on date (`inspection_date` column). With the oldest data in the train split, newer in validate, and most recent in test.
>
> Why split on time?
> This is to avoid leaking data between the dataset splits as this dataset is time-dependent, as the newer information better predicts the other newer rows than does the older rows. For example the `inspection_score` of a restaurant in June is likely closer to the July score than the January score.
>
> _... (preview of dataset [followed](https://github.com/dotnet/docs/issues/17721))..._
Issue also filed in docs repo to track that change -- https://github.com/dotnet/docs/issues/17962
Hướng dẫn đóng góp
Hướng nghiên cứu
Bắt đầu bằng cách đọc cuộc thảo luận được liên kết trong các issue 17721 và 17962 của dotnet/docs, sau đó xem tutorial tại docs/machine-learning/tutorials/health-violation-classification-model-builder.md và sample tại samples/modelbuilder/MulticlassClassification_RestaurantViolations. Thay thế dataset gây rò rỉ và căn chỉnh sample cùng tutorial theo task đã chọn, sử dụng các phần chia train, validation và test dựa trên thời gian như đã thảo luận; hoàn thành nghĩa là cả hai repository đều nhất quán sử dụng dataset mới.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- machine-learning
- Lĩnh vực
- documentation, machine-learning
- Loại issue
- Tính năng
- Độ khó
- 5/5
- Thời gian dự kiến
- Hơn một tuần
- Mức độ hoạt động
- Đình trệ
- Độ rõ ràng
- Khá rõ ràng
- Mức phù hợp với người mới
- 25/100