dotnet / dotnet/machinelearning
DataFrame doesn't decode boolean arrays correctly from Arrow
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
**System Information (please complete the following information):**
- OS & Version: Windows 10
- ML.NET Version: 0.21.1
- .NET Version: .NET 5.0
**Describe the bug**
Creating a dataframe from an arrow record batch where a column is a boolean array produces incorrect results (and occasionally even throws exceptions).
**To Reproduce**
Run:
```csharp
public void DataFrameDecodeBooleans()
{
BooleanArray boolArray = new BooleanArray.Builder().AppendNull().Append(false).Append(true).Build();
Field boolField = new Field("col", BooleanType.Default, true);
Schema schema = new Schema(new[] { boolField }, null);
RecordBatch batch = new RecordBatch(schema, new IArrowArray[] { boolArray }, boolArray.Length);
DataFrame df = DataFrame.FromArrowRecordBatch(batch);
DataFrameColumn boolCol = df["col"];
Assert.Equal(boolArray.Length, boolCol.Length);
for (int row = 0; row < boolArray.Length; row++)
{
Assert.Equal(boolArray.GetValue(row), boolCol[row]);
}
}
```
**Expected behavior**
Above test passes
Contributor guide
Assessment
This issue has not been assessed yet.