dotnet / dotnet/machinelearning
`ConvertToOnnx` should also accept `DataViewSchema`
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
**Is your feature request related to a problem? Please describe.**
Currently, [saving a model to zip file](https://learn.microsoft.com/en-us/dotnet/api/microsoft.ml.modeloperationscatalog.save?view=ml-dotnet) only requires a `DataViewSchema`, but [saving a model to ONNX](https://learn.microsoft.com/en-us/dotnet/api/microsoft.ml.onnxexportextensions.converttoonnx?view=ml-dotnet-preview) requires `IDataView`.
Inside [`ConvertToOnnxProtobufCore`](https://github.com/dotnet/machinelearning/blob/main/src/Microsoft.ML.OnnxConverter/OnnxExportExtensions.cs#L18), a prediction was performed (`transform.Transform(inputData)`), which may be expensive if the training data set is large.
**Describe the solution you'd like**
`ConvertToOnnx` should have overloads that accept `DataViewSchema`, then convert the `DataViewSchema` to an [empty `IDataView`](https://github.com/dotnet/machinelearning/blob/main/src/Microsoft.ML.Data/DataView/EmptyDataView.cs), and pass the empty `IDataView` to the methods accepting `IDataView`.
The performance of methods accepting `IDataView` may be improved if `EmptyDataView` is created from the [Schema of the IDataView](https://learn.microsoft.com/en-us/dotnet/api/microsoft.ml.idataview.schema?view=ml-dotnet) and passed to `ConvertToOnnxProtobuf`, instead of full data.
**Describe alternatives you've considered**
Nil
**Additional context**
I have implemented the proposed solution and that seems working well. It is unfortunate that [`EmptyDataView`](https://github.com/dotnet/machinelearning/blob/main/src/Microsoft.ML.Data/DataView/EmptyDataView.cs) is an `internal` class, so I have to implement my own `EmptyDataView`.
```cs
sealed record EmptyDataView(DataViewSchema Schema) : IDataView {
public bool CanShuffle => true;
public long? GetRowCount() => 0L;
public DataViewRowCursor GetRowCursor(IEnumerable columnsNeeded, Random? rand = null)
=> new EmptyDataViewRowCursor(Schema);
public DataViewRowCursor[] GetRowCursorSet(IEnumerable columnsNeeded, int n, Random? rand = null)
=> Array.Empty();
}
sealed class EmptyDataViewRowCursor : DataViewRowCursor {
private readonly DataViewSchema schema;
public EmptyDataViewRowCursor(DataViewSchema Schema) {
schema = Schema;
}
public override DataViewSchema Schema => schema;
public override long Position => -1L;
public override bool IsColumnActive(DataViewSchema.Column column) => false;
public override bool MoveNext() => false;
public override long Batch => 0L;
public override ValueGetter GetGetter(DataViewSchema.Column column)
=> throw new InvalidOperationException();
public override ValueGetter GetIdGetter()
=> throw new InvalidOperationException();
}
```
Contributor guide
Assessment
This issue has not been assessed yet.