dotnet / dotnet/machinelearning-modelbuilder

Remove '\n' and '\r' from datasets

Open
#1,008 4 comments 0 reactions 0 assignees View on GitHub
Code Example? Priority:2 Reported by: Customer
Dominant language
Dockerfile
Stars
285
Forks
66
PR merge metrics
No merged PRs in 30d

Description

System Information:
- Model Builder Version: 16.7.3
- Visual Studio Version: 16.1.0

The ML Builder fails at training part with these messages:
`
2020-09-21 16:28:34.7573 DEBUG Set log file path to C:\Users\Admin\AppData\Local\Temp\MLVSTools\logs\9a23c49b-556a-484f-bb0f-63101f9af44f.txt (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:28:44.0690 DEBUG C:\Users\Admin\Documents\datasets\spam dataset\spamFixed.csv (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:28:58.7341 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:28:59.6822 WARN GPU Service not found. Falling back to CPU AutoML Service. (Microsoft.ML.ModelBuilder.Utils.Logger.Warn)
2020-09-21 16:29:01.8903 DEBUG Unable to split the file provided into multiple, consistent columns.
at Microsoft.ML.AutoML.ColumnInferenceApi.InferSplit(MLContext context, TextFileSample sample, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse)
at Microsoft.ML.AutoML.ColumnInferenceApi.InferColumns(MLContext context, String path, ColumnInformation columnInfo, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse, Boolean trimWhitespace, Boolean groupColumns, Boolean hasHeader)
at Microsoft.ML.ModelBuilder.AutoMLEngine.InferColumns(MLContext context, AutoMLServiceParamater config, ColumnInformation columnInformation)
at Microsoft.ML.ModelBuilder.AutoMLEngine.d__30.MoveNext() in /_/src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLEngineService/AutoMLEngine.cs:line 112 (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:35:04.3941 DEBUG C:\Users\Admin\Documents\datasets\spam dataset\spamFixed.csv (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:35:13.1413 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:35:13.1783 DEBUG Unable to split the file provided into multiple, consistent columns.
at Microsoft.ML.AutoML.ColumnInferenceApi.InferSplit(MLContext context, TextFileSample sample, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse)
at Microsoft.ML.AutoML.ColumnInferenceApi.InferColumns(MLContext context, String path, ColumnInformation columnInfo, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse, Boolean trimWhitespace, Boolean groupColumns, Boolean hasHeader)
at Microsoft.ML.ModelBuilder.AutoMLEngine.InferColumns(MLContext context, AutoMLServiceParamater config, ColumnInformation columnInformation)
at Microsoft.ML.ModelBuilder.AutoMLEngine.d__30.MoveNext() in /_/src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLEngineService/AutoMLEngine.cs:line 112 (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:20.4254 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:20.4254 DEBUG Disposing AutoMLService Client (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:20.4254 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:20.4454 DEBUG The process cannot access the file 'C:\Users\Admin\Documents\datasets\spam dataset\spamFixed.csv' because it is being used by another process.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Microsoft.ML.AutoML.TextFileSample.CreateFromFullFile(String path)
at Microsoft.ML.AutoML.ColumnInferenceApi.InferColumns(MLContext context, String path, ColumnInformation columnInfo, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse, Boolean trimWhitespace, Boolean groupColumns, Boolean hasHeader)
at Microsoft.ML.ModelBuilder.AutoMLEngine.InferColumns(MLContext context, AutoMLServiceParamater config, ColumnInformation columnInformation)
at Microsoft.ML.ModelBuilder.AutoMLEngine.d__30.MoveNext() in /_/src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLEngineService/AutoMLEngine.cs:line 112 (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:24.3317 DEBUG Open Log FileC:\Users\Admin\AppData\Local\Temp\MLVSTools\logs\9a23c49b-556a-484f-bb0f-63101f9af44f.txt (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:48.1415 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:48.1455 DEBUG Disposing AutoMLService Client (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:48.1455 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:48.1725 DEBUG Unable to split the file provided into multiple, consistent columns.
at Microsoft.ML.AutoML.ColumnInferenceApi.InferSplit(MLContext context, TextFileSample sample, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse)
at Microsoft.ML.AutoML.ColumnInferenceApi.InferColumns(MLContext context, String path, ColumnInformation columnInfo, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse, Boolean trimWhitespace, Boolean groupColumns, Boolean hasHeader)
at Microsoft.ML.ModelBuilder.AutoMLEngine.InferColumns(MLContext context, AutoMLServiceParamater config, ColumnInformation columnInformation)
at Microsoft.ML.ModelBuilder.AutoMLEngine.d__30.MoveNext() in /_/src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLEngineService/AutoMLEngine.cs:line 112 (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:36:49.5316 DEBUG Open Log FileC:\Users\Admin\AppData\Local\Temp\MLVSTools\logs\9a23c49b-556a-484f-bb0f-63101f9af44f.txt (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:04.9753 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:04.9753 DEBUG Disposing AutoMLService Client (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:05.1113 DEBUG Disposing AutoMLService Client (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:05.1113 DEBUG Disposing AutoMLService Client (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:05.1113 DEBUG Disposing AutoMLService Client (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:15.6691 DEBUG C:\Users\Admin\Documents\datasets\spam dataset\spamFixed.csv (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:26.4699 DEBUG Disposing TrainSession (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:26.4929 DEBUG Unable to split the file provided into multiple, consistent columns.
at Microsoft.ML.AutoML.ColumnInferenceApi.InferSplit(MLContext context, TextFileSample sample, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse)
at Microsoft.ML.AutoML.ColumnInferenceApi.InferColumns(MLContext context, String path, ColumnInformation columnInfo, Nullable`1 separatorChar, Nullable`1 allowQuotedStrings, Nullable`1 supportSparse, Boolean trimWhitespace, Boolean groupColumns, Boolean hasHeader)
at Microsoft.ML.ModelBuilder.AutoMLEngine.InferColumns(MLContext context, AutoMLServiceParamater config, ColumnInformation columnInformation)
at Microsoft.ML.ModelBuilder.AutoMLEngine.d__30.MoveNext() in /_/src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLEngineService/AutoMLEngine.cs:line 112 (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2020-09-21 16:37:27.6030 DEBUG Open Log FileC:\Users\Admin\AppData\Local\Temp\MLVSTools\logs\9a23c49b-556a-484f-bb0f-63101f9af44f.txt (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
`

Steps to reproduce the behavior:
Train a Text Classifier using this dataset: https://www.kaggle.com/uciml/sms-spam-collection-dataset

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.