dotnet / dotnet/machinelearning

Object detection training using CUDA gives decent results, while using CPU always returns nothing or inferior results

Open
#7,028 0 comments 0 reactions 0 assignees View on GitHub
area-Vision untriaged
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

**System Information (please complete the following information):**

- OS & Version: Windows 11
- ML.NET Version: 16.18.2
- .NET Version: 8.0

**Describe the bug**
I have a self created training set where I want to detect small marked passages of text (with a text marker). If I use CUDA for object detection training I get like the following result (using these parameters: --epoch 10 --device gpu0 -b 1 -st 0.3 -it 0.5 --width 600 --height 800)

-> ObjectDetectionMulti 0.6115

If I switch to CPU training while keeping all other parameters the result is always 0.0%. Testing the CPU model with some test images will never return any boxes, while the model from GPU training generates nice hits.
As a workaround I temporarily use the GPU model with CPU code.

I know what I try to do is a bit strange, but it seems to work out if the GPU is used for training. A similar thing happens when I use the stop sign tutorial data set. Results when using the CPU for 5 epochs gives approx. 0.53 as result, while using the GPU on the same set with the same parameters 0.63.

I added my dataset for further analysis, all used documents are public available on the internet.

[LetterMarkerAnnot.zip](https://github.com/dotnet/machinelearning/files/14393399/LetterMarkerAnnot.zip)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.