dotnet / dotnet/machinelearning

OneHotEncoding with Outputkind.Bin spits out wrong dimension data arrays

Open
#2,774 0 comments 0 reactions 1 assignee Claimed by @codemzs View on GitHub
area-Transforms bug Priority:1
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

### System information

- **OS version/distro**: 0.11.0-preview-27427-9

### Issue
OneHotEncoding with Outputkind.Bin spits out data elements of 3-dimensional arrays, while 2 dimensional arrays expected. Outputkind.Ind and Outputkind.Bag are correctly spitting out 2 dimensional arrays. See repro source code below

### Source code / logs
using System;
using System.Linq;
using Microsoft.ML;
using Microsoft.ML.Data;
using Microsoft.ML.Transforms.Categorical;

namespace ConsoleApp1
{
class TestData
{
public string data;
}

class Program
{
static void Main(string[] args)
{
var trainData = new[] { new TestData() { data = "male" }, new TestData() { data = "female"} };
var testData = new[] { new TestData() { data = "male" }, new TestData() { data = "female" }, new TestData() { data = "fem" }, new TestData() { data = "fem1" } };

var mlContext = new MLContext();
var trainDataView = mlContext.Data.LoadFromEnumerable(trainData);
var testDataView = mlContext.Data.LoadFromEnumerable(testData);

var pipe = mlContext.Transforms.Categorical.OneHotEncoding("Bag", "data", OneHotEncodingTransformer.OutputKind.Bag)
.Append(mlContext.Transforms.Categorical.OneHotEncoding("Key", "data", OneHotEncodingTransformer.OutputKind.Key))
.Append(mlContext.Transforms.Categorical.OneHotEncoding("Ind", "data", OneHotEncodingTransformer.OutputKind.Ind))
.Append(mlContext.Transforms.Categorical.OneHotEncoding("Bin", "data", OneHotEncodingTransformer.OutputKind.Bin));

var transformer = pipe.Fit(trainDataView);

var result = transformer.Transform(testDataView);

var bags = result.GetColumn(mlContext, "Bag").ToArray();
var inds = result.GetColumn(mlContext, "Ind").ToArray();
var bins = result.GetColumn(mlContext, "Bin").ToArray();

Console.WriteLine("Number of dimensions (should be 2) in each output kind: {0} {1} {2}", bags[0].Length, inds[0].Length, bins[0].Length);
// Number of dimensions (should be 2) in each output kind: 2 2 3
}
}
}

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.