dotnet / dotnet/machinelearning-modelbuilder

Object detection locally stuck in training phase on tutorial dataset

Abierto
#2,887 4 comentarios 0 reacciones 0 asignados Ver en GitHub
GS Tutorial
Lenguaje dominante
Dockerfile
Estrellas
285
Forks
66
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

**System Information (please complete the following information):**
- Model Builder or CLI Version: 17.18.2.2415501
- Visual Studio Version (if applicable): 2022 version 17.9.3

**Describe the bug**
- On which page of the tutorial did you run into an issue (URL): https://devblogs.microsoft.com/dotnet/object-detection-ml-dotnet-model-builder/ (Step 5 of topic "Get started with object detection locally")
- Clear description of the problem: Training locally gets stuck in the training stage, waited about 6 hours, tried CPU (Ryzen 5 7600X with 32GB ram) or GPU (RTX 3070) environment, nothing changes

**Logs**
```
2024-03-18 22:32:57.5620 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:32:57.5620 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:32:58.5660 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:32:58.5660 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:32:59.5760 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:32:59.5760 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:00.5960 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:00.5960 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:01.6030 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:01.6030 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:02.6175 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:02.6175 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:03.6262 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:03.6262 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:04.6287 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:04.6287 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:05.6395 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:05.6395 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:06.6409 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:06.6409 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:07.6492 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:07.6492 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:08.6499 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:08.6499 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:09.6540 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:09.6540 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:10.6595 DEBUG update TrainViewModel_159.RemainingSeconds (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
2024-03-18 22:33:10.6595 DEBUG update TrainViewModel_159.RemainingTime (Microsoft.ML.ModelBuilder.Utils.Logger.Debug)
```

**To Reproduce**
Steps to reproduce the behavior: "Get started with object detection locally" https://devblogs.microsoft.com/dotnet/object-detection-ml-dotnet-model-builder/

**Expected behavior**
With my hardware spec and 40 images with one category detection train phase should take a few minutes

**Screenshots**
![image](https://github.com/dotnet/machinelearning-modelbuilder/assets/10048426/520ea5d7-b68a-453c-924c-9a69b0ef5477)
![image](https://github.com/dotnet/machinelearning-modelbuilder/assets/10048426/5654e88d-59ea-4a7b-97b0-1157ac10312f)

**Additional context**
Classification scenario works as expected locally with same dataset

![image](https://github.com/dotnet/machinelearning-modelbuilder/assets/10048426/ef975ebf-166f-4c64-a030-7e784dd89ca4)

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Línea de trabajo

Comienza con el paso 5 del tutorial enlazado “Get started with object detection locally” y reproduce la fase de entrenamiento usando la versión de Model Builder y el dataset indicados. Rastrea las actualizaciones repetidas de los registros TrainViewModel_159 RemainingSeconds y RemainingTime y, después, compara las ejecuciones en CPU y GPU. Se considera terminado cuando el dataset del tutorial completa el entrenamiento en los pocos minutos esperados o se informa del fallo con una causa clara.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
machine-learning
Área
machine-learning
Tipo de issue
Error
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
25/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.