dotnet / dotnet/machinelearning
ImagePixelExtractingEstimator should not require the ImageResizingEstimator in the pipeline
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
Currently in ML.Net, to be able to use an ImagePixelExtractingTransformer/ImagePixelExtractingEstimator, you need to have an ImageResizingEstimator in the pipeline before the ImagePixelExtracting, otherwise the pipeline [fails at run-time ](https://github.com/dotnet/machinelearning/blob/master/src/Microsoft.ML.ImageAnalytics/ImagePixelExtractorTransform.cs#L447)with a schema mismatch error.
`"'Schema mismatch for input column 'ImageObject': expected known-size image, got unknown-size image'". `
This happen because the ImagePixelExtracting operating on [ImageTypes ](https://github.com/dotnet/machinelearning/blob/master/src/Microsoft.ML.ImageAnalytics/ImageType.cs)needs the Width and the Height properties, to extract the pixels.
The Estimator that creates the [ImageType ](https://github.com/dotnet/machinelearning/blob/master/src/Microsoft.ML.ImageAnalytics/ImageType.cs)object, is the ImageLoadingEstimator, but the Width and Height properties of the estimator don't get filled until they get passed to ImageResizing.
Proposed fix:
- The ImageLoadingEstimator should allow the user to specify the height and width of the images, if the user has images of fixed with and height, rather than fail at "ExtractPixels" if resize is not present.
- if we make a bool about `imagesHaveSameDimentions`, and that is set to `yes`, the ImageLoadingTransformer could memorize the width and height of the first image, and check every subsequent image, drop all the samples that don't have the right dimensions, and warn about the records dropped.
cc @yaeldekel @Ivanidzo4ka @TomFinley
Contributor guide
Assessment
This issue has not been assessed yet.