dotnet / dotnet/machinelearning-samples
Review samples for correct data science approach and ML.NET API usage
- Dominant language
- PowerShell
- Stars
- 4.7k
- Forks
- 2.7k
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 1
Description
# Status
| Folder | Sample | Data Science Review | API Review |
| ------------- | ------------- |------------- |------------- |
|C#\getting_started | BinaryClassification_ CreditCardFraudDetection | OK (#96) | OK (v0.8) |
| C#\getting_started | BinaryClassification_ SentimentAnalysis | OK | OK (v0.8) |
| C#\getting_started | Clustering_ CustomerSegmentation | OK (#95) | OK (v0.8) |
| C#\getting_started | Clustering_ Iris | OK (#109) | OK (v0.8) |
| C#\getting_started | MulticlassClassification_ Iris | OK | OK (v0.8) |
| C#\getting_started | Regression_ BikeSharingDemand | OK | OK (v0.8) |
| C#\getting_started | Regression_ TaxiFarePrediction | OK (#95) | OK (v0.8) |
| C#\getting_started | DeepLearning_ ImageClassification_ TensorFlow | **Pixel data preprocessing needed or not? Also, info/github page for the used inception model should be included** | OK (v0.8) #155 |
| C#\getting_started | DeepLearning_ TensorFlowEstimator | **Same as above** | **Still 0.7** |
| C#\getting_started | MatrixFactorization_ MovieRecommendation | OK | **Still 0.7** |
| C#\end-to-end-apps | MulticlassClassification- GitHubLabeler | OK (#96) | OK (v0.8) |
| C#\end-to-end-apps | Recommendation- MovieRecommender | OK | **Still 0.7** |
| C#\end-to-end-apps | Regression- SalesForecast | OK | OK (v0.8) |
| C#\getting_started | AnomalyDetection- Sales | OK | OK (v0.11) |
# DS Review
## BinaryClassification_CreditCardFraudDetection
- Data preprocessing: OK (MeanVar normalization)
- Feature engineering: Not needed (input data are PCA dimensions)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics: Ok
- Accuracy: 99.9%
- Auc: 97.5%
- F1Score: 77.7%
## BinaryClassification_SentimentAnalysis
- Data preprocessing: Not needed
- Feature engineering: Ok (Text -> Feature vector using TextTransform)
- Learner: Ok
- Training: Ok (single training on training data)
- Scoring: Ok
- Metrics: Ok
- Accuracy: 72%
- Auc: 97%
- F1Score: 78%
## Clustering_CustomerSegmentation
- Data preprocessing: Ok (Join and pivot tables using Linq)
- Feature engineering: Ok (Counting offers per customers)
- Learner: Ok
- Training: Why PcaEstimator seed is 42 (magic number)?
- Scoring: Ok
- Metrics: Ok
- AvgMinScore: 2.3
- Dbi: 2.5
## Clustering_Iris
- Data loading: Ok (Can load all the numeric values as vector instead of individually and then concatenating them. But keeping it as is for educational purposes.)
- Data preprocessing: Ok (not needed)
- Feature engineering: Ok (not needed)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics:
- AvgMinScore: 0.564
- DBI: 0.955
## MulticlassClassification_Iris
- Data loading: Ok (Can load all the numeric values as vector instead of individually and then concatenating them. But keeping it as is for educational purposes.)
- Data preprocessing: Ok (not needed)
- Feature engineering: Ok (not needed)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics: Ok (Accuracy is 1 because of small test set)
## Regression_BikeSharingDemand
Ok.
## Regression_TaxiFarePrediction
- Data loading: Ok.
- Data preprocessing: Ok (not needed)
- Feature engineering: Ok (Categorical transform for text columns)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics: LossFn needs to be removed from outputted metrics. It's the same as L2 b/c no custom loss function is defined.
- R2 Score: 0.7
- RMS loss: 5.97
- Absolute loss: .99
## MatrixFactorization_MovieRecommendation
MF using MFTrainer. Evaluation done as regressions.
## MulticlassClassification-GitHubLabeler
- Data loading: Ok.
- Data preprocessing: Ok (not needed)
- Feature engineering: Ok (Categorical transform for text columns)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics: Ok
- MicroAcuracy (Avg): 71.8%
- MacroAccuracy (Avg): 50.3%
- LogLoss: 1.076
- LogLossReduction: 56.2
## Regression-SalesForecast (eShopDashboardML)
- Data loading: Ok.
- Data preprocessing: Ok (not needed)
- Feature engineering: Ok (Categorical transform for text columns)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics: Ok
- Product model:
- L1 Loss: 96.5
- L2 Loss: 74493.7
- RMS: 96.5
- R-squared: 56.6%
- Country model:
- L1 Loss: 0.446
- L2 Loss: 0.386
- RMS: 0.446
- R-squared: 45.3%
## AnomalyDetection-Sales
- Data loading: Ok.
- Data preprocessing: Ok (not needed)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics: there is no evaluation in timeseries- spike detection and change point detection algorithm
Contributor guide
Assessment
This issue has not been assessed yet.