dotnet / dotnet/machinelearning

Problems with convolutional neural networks loaded from Keras model

Open
#6,017 0 comments 0 reactions 0 assignees View on GitHub
area-TensorFlow
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

**System Information (please complete the following information):**
- OS & Version: Windows 10
- ML.NET Version: 1.6.0
- .NET Version: .NET Framework 4.7.1

**Describe the bug**
When attempting to load convolutional neural networks, with a non predetermined size required for input (*,*,1 size), we encounter the following problems:
- When we attempt to run the network on a (n,1,1) sized input, only n values which are dividable by 4 are accepted, although the network should be capable of processing such an input, the following exception is thrown:
`System.InvalidOperationException: 'Input shape mismatch: Input 'serving_default_input_6' has shape (None, None, 1), but input data is of length 257.`

- Even when the input is accepted, the results are not equivalent to the ones resulted from keras, seemingly smaller parts of the input is considered input for the network (see later in screenshots)

**Expected behavior**
Same behaviour to the Keras environment

**Screenshots, Code, Sample Projects**
Code for reproduction of the bugs:

```C#
using System;
using System.Linq;
using Microsoft.ML;
using Microsoft.ML.Data;
using System.IO;

namespace KerasMLNET_demo
{
class Program
{
static void Main(string[] args)
{

//loading the trained model
string modellocation = @"./model/noisefilter";
var mlContext = new MLContext();
using var tensorFlowModel = mlContext.Model.LoadTensorFlowModel(modellocation);
var inputschema = tensorFlowModel.GetInputSchema();
var inputlayer_ID = inputschema[0].Name;

//generating input data:
var rand = new Random();
float[] inputdata = new float[256];
for (int i = 0; i < inputdata.Length; i++)
{
inputdata[i] = (float)(rand.NextDouble() + 25*Math.Exp(-0.025*((128.0 - i) * (128.0 - i))));
}
File.WriteAllLines("input.csv", inputdata.Select(d => d.ToString(System.Globalization.CultureInfo.CreateSpecificCulture("en-US"))));
var datalist = new SpectrumCollection[]
{
new SpectrumCollection(){Spectrum = inputdata }
};
var dataView = mlContext.Data.LoadFromEnumerable(datalist);

//generating ML model from the keras network
var pipeline = mlContext.Transforms.CopyColumns(inputlayer_ID, "Spectrum").Append(tensorFlowModel.ScoreTensorFlowModel("StatefulPartitionedCall", inputlayer_ID)).Append(mlContext.Transforms.CopyColumns("Spectrum", "StatefulPartitionedCall"));
ITransformer model = pipeline.Fit(dataView);
var engine = mlContext.Model.CreatePredictionEngine(model);
// Predict with TensorFlow pipeline.
SpectrumCollection prediction;
float[] PredSpectrum;
datalist[0].Spectrum = inputdata;
prediction = engine.Predict(datalist[0]);
PredSpectrum = prediction.Spectrum;
File.WriteAllLines("prediction.csv", PredSpectrum.Select(d => d.ToString(System.Globalization.CultureInfo.CreateSpecificCulture("en-US"))));
}

public class SpectrumCollection
{
[VectorType(256, 1, 1)]
[ColumnName("Spectrum")]
public float[] Spectrum { get; set; }
}

}
}
```
Screenshot with expected results:

![pythonVSmlnet](https://user-images.githubusercontent.com/95739641/145171969-f2ed4a87-b252-4de3-a0df-281a71ad8f52.png)

Model for reproduction:

[model.zip](https://github.com/dotnet/machinelearning/files/7674358/model.zip)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.