dotnet / dotnet/machinelearning-samples

Review samples for correct data science approach and ML.NET API usage

Open
#81 15 comments 0 reactions 0 assignees View on GitHub
Dominant language
PowerShell
Stars
4.7k
Forks
2.7k
Avg merge
2d 22h
Merged PRs (30d)
1

Description

# Status
| Folder | Sample | Data Science Review | API Review |
| ------------- | ------------- |------------- |------------- |
|C#\getting_started | BinaryClassification_ CreditCardFraudDetection | OK (#96) | OK (v0.8) |
| C#\getting_started | BinaryClassification_ SentimentAnalysis | OK | OK (v0.8) |
| C#\getting_started | Clustering_ CustomerSegmentation | OK (#95) | OK (v0.8) |
| C#\getting_started | Clustering_ Iris | OK (#109) | OK (v0.8) |
| C#\getting_started | MulticlassClassification_ Iris | OK | OK (v0.8) |
| C#\getting_started | Regression_ BikeSharingDemand | OK | OK (v0.8) |
| C#\getting_started | Regression_ TaxiFarePrediction | OK (#95) | OK (v0.8) |
| C#\getting_started | DeepLearning_ ImageClassification_ TensorFlow | **Pixel data preprocessing needed or not? Also, info/github page for the used inception model should be included** | OK (v0.8) #155 |
| C#\getting_started | DeepLearning_ TensorFlowEstimator | **Same as above** | **Still 0.7** |
| C#\getting_started | MatrixFactorization_ MovieRecommendation | OK | **Still 0.7** |
| C#\end-to-end-apps | MulticlassClassification- GitHubLabeler | OK (#96) | OK (v0.8) |
| C#\end-to-end-apps | Recommendation- MovieRecommender | OK | **Still 0.7** |
| C#\end-to-end-apps | Regression- SalesForecast | OK | OK (v0.8) |
| C#\getting_started | AnomalyDetection- Sales | OK | OK (v0.11) |

# DS Review
## BinaryClassification_CreditCardFraudDetection
- Data preprocessing: OK (MeanVar normalization)
- Feature engineering: Not needed (input data are PCA dimensions)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics: Ok
- Accuracy: 99.9%
- Auc: 97.5%
- F1Score: 77.7%

## BinaryClassification_SentimentAnalysis
- Data preprocessing: Not needed
- Feature engineering: Ok (Text -> Feature vector using TextTransform)
- Learner: Ok
- Training: Ok (single training on training data)
- Scoring: Ok
- Metrics: Ok
- Accuracy: 72%
- Auc: 97%
- F1Score: 78%

## Clustering_CustomerSegmentation
- Data preprocessing: Ok (Join and pivot tables using Linq)
- Feature engineering: Ok (Counting offers per customers)
- Learner: Ok
- Training: Why PcaEstimator seed is 42 (magic number)?
- Scoring: Ok
- Metrics: Ok
- AvgMinScore: 2.3
- Dbi: 2.5

## Clustering_Iris
- Data loading: Ok (Can load all the numeric values as vector instead of individually and then concatenating them. But keeping it as is for educational purposes.)
- Data preprocessing: Ok (not needed)
- Feature engineering: Ok (not needed)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics:
- AvgMinScore: 0.564
- DBI: 0.955

## MulticlassClassification_Iris
- Data loading: Ok (Can load all the numeric values as vector instead of individually and then concatenating them. But keeping it as is for educational purposes.)
- Data preprocessing: Ok (not needed)
- Feature engineering: Ok (not needed)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics: Ok (Accuracy is 1 because of small test set)

## Regression_BikeSharingDemand
Ok.

## Regression_TaxiFarePrediction
- Data loading: Ok.
- Data preprocessing: Ok (not needed)
- Feature engineering: Ok (Categorical transform for text columns)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics: LossFn needs to be removed from outputted metrics. It's the same as L2 b/c no custom loss function is defined.
- R2 Score: 0.7
- RMS loss: 5.97
- Absolute loss: .99

## MatrixFactorization_MovieRecommendation
MF using MFTrainer. Evaluation done as regressions.

## MulticlassClassification-GitHubLabeler
- Data loading: Ok.
- Data preprocessing: Ok (not needed)
- Feature engineering: Ok (Categorical transform for text columns)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics: Ok
- MicroAcuracy (Avg): 71.8%
- MacroAccuracy (Avg): 50.3%
- LogLoss: 1.076
- LogLossReduction: 56.2

## Regression-SalesForecast (eShopDashboardML)
- Data loading: Ok.
- Data preprocessing: Ok (not needed)
- Feature engineering: Ok (Categorical transform for text columns)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics: Ok
- Product model:
- L1 Loss: 96.5
- L2 Loss: 74493.7
- RMS: 96.5
- R-squared: 56.6%
- Country model:
- L1 Loss: 0.446
- L2 Loss: 0.386
- RMS: 0.446
- R-squared: 45.3%

## AnomalyDetection-Sales
- Data loading: Ok.
- Data preprocessing: Ok (not needed)
- Learner: Ok
- Training: Ok
- Scoring: Ok
- Metrics: there is no evaluation in timeseries- spike detection and change point detection algorithm

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.