dotnet / dotnet/machinelearning-modelbuilder
Remove '\n' and '\r' from datasets
- Lingua principale
- Dockerfile
- Stelle
- 285
- Fork
- 66
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
System Information:
- Model Builder Version: 16.7.3
- Visual Studio Version: 16.1.0
The ML Builder fails at training part with these messages:
`
2020-09-21 16:28:34.7573 DEBUG Set log file path to C:\Users\Admin\AppData\Local\Temp\MLVSTools\logs\9a23c49b-556a-484f-bb0f-63101f9af44f.txt (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:28:44.0690 DEBUG C:\Users\Admin\Documents\datasets\spam dataset\spamFixed.csv (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:28:58.7341 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:28:59.6822 WARN GPU Service not found. Falling back to CPU AutoML Service. (Microsoft.ML.ModelBuilder.Utils.Logger.Warn)
2020-09-21 16:29:01.8903 DEBUG Unable to split the file provided into multiple, consistent columns.
at Microsoft.ML.AutoML.ColumnInferenceApi.InferSplit(MLContext context, TextFileSample sample, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse)
at Microsoft.ML.AutoML.ColumnInferenceApi.InferColumns(MLContext context, String path, ColumnInformation columnInfo, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse, Boolean trimWhitespace, Boolean groupColumns, Boolean hasHeader)
at Microsoft.ML.ModelBuilder.AutoMLEngine.InferColumns(MLContext context, AutoMLServiceParamater config, ColumnInformation columnInformation)
at Microsoft.ML.ModelBuilder.AutoMLEngine.d__30.MoveNext() in /_/src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLEngineService/AutoMLEngine.cs:line 112 (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:35:04.3941 DEBUG C:\Users\Admin\Documents\datasets\spam dataset\spamFixed.csv (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:35:13.1413 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:35:13.1783 DEBUG Unable to split the file provided into multiple, consistent columns.
at Microsoft.ML.AutoML.ColumnInferenceApi.InferSplit(MLContext context, TextFileSample sample, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse)
at Microsoft.ML.AutoML.ColumnInferenceApi.InferColumns(MLContext context, String path, ColumnInformation columnInfo, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse, Boolean trimWhitespace, Boolean groupColumns, Boolean hasHeader)
at Microsoft.ML.ModelBuilder.AutoMLEngine.InferColumns(MLContext context, AutoMLServiceParamater config, ColumnInformation columnInformation)
at Microsoft.ML.ModelBuilder.AutoMLEngine.d__30.MoveNext() in /_/src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLEngineService/AutoMLEngine.cs:line 112 (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:20.4254 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:20.4254 DEBUG Disposing AutoMLService Client (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:20.4254 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:20.4454 DEBUG The process cannot access the file 'C:\Users\Admin\Documents\datasets\spam dataset\spamFixed.csv' because it is being used by another process.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Microsoft.ML.AutoML.TextFileSample.CreateFromFullFile(String path)
at Microsoft.ML.AutoML.ColumnInferenceApi.InferColumns(MLContext context, String path, ColumnInformation columnInfo, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse, Boolean trimWhitespace, Boolean groupColumns, Boolean hasHeader)
at Microsoft.ML.ModelBuilder.AutoMLEngine.InferColumns(MLContext context, AutoMLServiceParamater config, ColumnInformation columnInformation)
at Microsoft.ML.ModelBuilder.AutoMLEngine.d__30.MoveNext() in /_/src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLEngineService/AutoMLEngine.cs:line 112 (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:24.3317 DEBUG Open Log FileC:\Users\Admin\AppData\Local\Temp\MLVSTools\logs\9a23c49b-556a-484f-bb0f-63101f9af44f.txt (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:48.1415 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:48.1455 DEBUG Disposing AutoMLService Client (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:48.1455 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:48.1725 DEBUG Unable to split the file provided into multiple, consistent columns.
at Microsoft.ML.AutoML.ColumnInferenceApi.InferSplit(MLContext context, TextFileSample sample, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse)
at Microsoft.ML.AutoML.ColumnInferenceApi.InferColumns(MLContext context, String path, ColumnInformation columnInfo, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse, Boolean trimWhitespace, Boolean groupColumns, Boolean hasHeader)
at Microsoft.ML.ModelBuilder.AutoMLEngine.InferColumns(MLContext context, AutoMLServiceParamater config, ColumnInformation columnInformation)
at Microsoft.ML.ModelBuilder.AutoMLEngine.d__30.MoveNext() in /_/src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLEngineService/AutoMLEngine.cs:line 112 (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:49.5316 DEBUG Open Log FileC:\Users\Admin\AppData\Local\Temp\MLVSTools\logs\9a23c49b-556a-484f-bb0f-63101f9af44f.txt (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:04.9753 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:04.9753 DEBUG Disposing AutoMLService Client (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:05.1113 DEBUG Disposing AutoMLService Client (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:05.1113 DEBUG Disposing AutoMLService Client (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:05.1113 DEBUG Disposing AutoMLService Client (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:15.6691 DEBUG C:\Users\Admin\Documents\datasets\spam dataset\spamFixed.csv (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:26.4699 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:26.4929 DEBUG Unable to split the file provided into multiple, consistent columns.
at Microsoft.ML.AutoML.ColumnInferenceApi.InferSplit(MLContext context, TextFileSample sample, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse)
at Microsoft.ML.AutoML.ColumnInferenceApi.InferColumns(MLContext context, String path, ColumnInformation columnInfo, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse, Boolean trimWhitespace, Boolean groupColumns, Boolean hasHeader)
at Microsoft.ML.ModelBuilder.AutoMLEngine.InferColumns(MLContext context, AutoMLServiceParamater config, ColumnInformation columnInformation)
at Microsoft.ML.ModelBuilder.AutoMLEngine.d__30.MoveNext() in /_/src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLEngineService/AutoMLEngine.cs:line 112 (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:27.6030 DEBUG Open Log FileC:\Users\Admin\AppData\Local\Temp\MLVSTools\logs\9a23c49b-556a-484f-bb0f-63101f9af44f.txt (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
`
Steps to reproduce the behavior:
Train a Text Classifier using this dataset: https://www.kaggle.com/uciml/sms-spam-collection-dataset
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Direzione di ricerca
Riproducete il malfunzionamento con il dataset Kaggle collegato sugli SMS spam, quindi iniziate dalla riga 112 di AutoMLService/AutoMLEngineService/AutoMLEngine.cs e seguite la chiamata di inferenza delle colonne fino al percorso InferColumns di ML.NET. Verificate che i caratteri di ritorno a capo e di nuova riga nel dataset non causino più errori di colonne incoerenti e che l’addestramento del classificatore di testo venga completato.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- csharp, machine-learning
- Ambito
- data, machine-learning
- Tipo di issue
- Bug
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100