dotnet / dotnet/iot

Update Iot.Device.Media to use Span<T> where possible.

Open
#1,727 4 comments 0 reactions 0 assignees View on GitHub
api-suggestion area-displays blocked Priority:3
Dominant language
C#
Stars
2.4k
Forks
630
Avg merge
11d 3h
Merged PRs (30d)
2

Description

While I was investigating color issues with YUYV data for #1726 I noticed that implementations in [VideoDevice.Converter.cs](https://github.com/dotnet/iot/blob/main/src/devices/Media/VideoDevice/VideoDevice.Converter.cs) could benefit from utilizing `Span` to speed up processing.

My initial tests just after updating `VideoDevice.YuyvToRgb` and `VideoDevice.RgbToBitmap` showed 200% improved processing time on a Raspberry PI 3B+ when capturing a YUYV image and converting it to a bitmap.

I believe (but haven't tested) that changing some of the internal functions in [UnixVideoDevice.cs](https://github.com/dotnet/iot/blob/main/src/devices/Media/VideoDevice/Devices/UnixVideoDevice.cs) to use `Span` instead of `byte[]` may net even more speed improvements, however it would be great if the `Capture` method returned a `Span` (as well as the `ImageBuffer` property on `NewImageBufferReadyEventArgs`).

I can certainly open a PR with the VideoDevice.Converter.cs improvements, but would it be acceptable to revisit the public API for `VideoDevice` to take full advantage of `Span`?

Updated VideoDevice.Converter.cs code from my testing:
```csharp
public static Color[] YuyvToRgb(Stream stream)
{
Span colorData = new YUYV[stream.Length / Unsafe.SizeOf()];
var colorDataBuffer = MemoryMarshal.Cast(colorData);
stream.Read(colorDataBuffer);

Color[] colors = new Color[colorData.Length * 2];
int i = 0;
foreach (var value in colorData)
{
colors[i++] = YuvToRgb(value.Y0, value.U, value.V);
colors[i++] = YuvToRgb(value.Y1, value.U, value.V);
}

return colors;
}

public static Bitmap RgbToBitmap((uint Width, uint Height) size, Color[] colors, System.Drawing.Imaging.PixelFormat format = System.Drawing.Imaging.PixelFormat.Format24bppRgb)
{
int width = (int)size.Width, height = (int)size.Height;
Bitmap pic = new Bitmap(width, height, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
Span bgrData = new Span(colors
.Select(c => new BGR()
{
R = c.R,
G = c.G,
B = c.B,
}).ToArray());
var imageData = pic.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.WriteOnly, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
unsafe
{
var imageDataSpan = new Span(imageData.Scan0.ToPointer(), imageData.Width * imageData.Height * 3);
bgrData.CopyTo(imageDataSpan);
}
pic.UnlockBits(imageData);

return pic;
}

private struct YUYV
{
public byte Y0;
public byte U;
public byte Y1;
public byte V;
}
private struct BGR
{
public byte B;
public byte G;
public byte R;
}
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.