dotnet / dotnet/machinelearning

Object Detection - YoloV3 - Corrupted PredictionResult

Open
#5,810 2 comments 0 reactions 0 assignees View on GitHub
area-ONNX Priority:3 question
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

### System information

- **OS Windows 10**:
- **.NET 5.0**:

### Issue

- **Trained a yolov3 object detector**
- **The prediction output is wrong and also does not change by changing the input image**

### Source code / logs

`
var context = new MLContext();

var emptyData = new List();

var data = context.Data.LoadFromEnumerable(emptyData);

var pipeline = context.Transforms.ResizeImages(
resizing: ImageResizingEstimator.ResizingKind.Fill,
outputColumnName: "feed_resized",
imageWidth: ImageSettings.imageWidth,
imageHeight: ImageSettings.imageHeight,
inputColumnName: nameof(CountInput.Image)
)
.Append(context.Transforms.ConvertToGrayscale(outputColumnName: "grayscale", inputColumnName: "feed_resized"))
.Append(context.Transforms.ExtractPixels(
outputColumnName: "input_1:0", inputColumnName: "grayscale",
interleavePixelColors: false,
colorsToExtract: Microsoft.ML.Transforms.Image.ImagePixelExtractingEstimator.ColorBits.Alpha,
outputAsFloatArray: true,
scaleImage: .00390625f))
.Append(context.Transforms.ApplyOnnxModel(
modelFile: _model_path,
outputColumnNames: new[] { "tf.concat_3", "tf.concat_5", "tf.concat_7" },
inputColumnNames: new[] { "input_1:0" })
);

var model = pipeline.Fit(data);

_predictionEngine = context.Model.CreatePredictionEngine(model);

string image_path = @"D:\ml_dotnet\tests\test.png";
var image = (Bitmap)Image.FromFile(image_path);

var prediction = _predictionEngine.Predict(new CountInput { Image = image });

my assumption is that the data is somehow not correctly transported from the ExtractPixels transformation to the ApplyOnnXModel transformation, that would at least explain why the result always stays the same.

The model has been converted from tensorflow2.

Link to my model: [https://drive.google.com/drive/folders/10n6J7Z0NzNTcTCV6dJbzGKoaRZDmWfbo?usp=sharing](url)

I already had a look in it with netron, but looks good to me.
![image](https://user-images.githubusercontent.com/26538077/119698910-0119dd00-be52-11eb-9e55-48e7e82f9556.png)

I am fairly new to this so bear with me in case anything is missing or I did some obvious mistakes.
I also went through the tutorials (especially regarding object detection), the tutorials itself ran fine but I am still missing something.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.