dotnet / dotnet/machinelearning-modelbuilder

model builder might produce erroneous models due to not able to specify `allowQuoting`.

Open
#2,648 8 comments 0 reactions 1 assignee Claimed by @LittleLittleCloud View on GitHub
Feature request Stale
Dominant language
Dockerfile
Stars
285
Forks
66
PR merge metrics
No merged PRs in 30d

Description

**System Information (please complete the following information):**
- Model Builder Version (available in Manage Extensions dialog): 17.14.4.2312404
- Visual Studio Version: 17.6.0

**Describe the bug**
- On which step of the process did you run into an issue: Data
- Clear description of the problem: there is nowhere to specify `allowQuoting` when loading from a text file so the generated code is

```
public static IDataView LoadIDataViewFromFile(MLContext mlContext, string inputDataFilePath, char separatorChar, bool hasHeader)
{
return mlContext.Data.LoadFromTextFile(inputDataFilePath, separatorChar, hasHeader);
}
```
The `allowQuoting` parameter for `LoadFromTextFile()` method is `false` by default so the text read in will be wrong if the input text file do include separators within double-quoted values. However, the model builder issues no warnings in such case and completes the training with an erroneous model.

**To Reproduce**
Steps to reproduce the behavior:
Just modify some of the text classification model builder tutorials by including separators within double-quoted text values and putting the label column after the feature columns. The label column will be messed up due to the separators within double-quoted text values.

**Expected behavior**
A clear and concise description of what you expected to happen.
* add an option to specify `allowQuoting` at the Data step and
* issue a warning/error if it is set to `false` but the input does include separators within double-quoted values.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Additional context**
Add any other context about the problem here.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.