deeplearning4j / deeplearning4j/deeplearning4j-examples
NaNs during training when running the TinyYoloHouseNumberDetection example
- Dominant language
- Java
- Stars
- 2.5k
- Forks
- 1.8k
- PR merge metrics
- No merged PRs in 30d
Description
#### Issue Description
Please describe your issue, along with:
- expected behavior: the network trains normally, no NaNs are reported
- encountered behavior: training crashes after the first fit() call because of NaNs (I enabled NaN detection, see below)
#### Version Information
Please indicate relevant versions, including, if relevant:
* Deeplearning4j version: 1.0.0-beta7 and 1.0.0-M2.1
* platform information: macOS Sonoma 14.2.1
The mentioned example:
https://github.com/deeplearning4j/deeplearning4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/advanced/modelling/objectdetection/TinyYoloHouseNumberDetection.java
You'll need to add this as the first call in the main() method:
```java
Nd4j.getExecutioner().setProfilingConfig(ProfilerConfig.builder()
.checkForINF(true)
.checkForNAN(true)
.checkElapsedTime(true)
.checkLocality(true)
.checkWorkspaces(true)
.build());
```
Contributor guide
Assessment
This issue has not been assessed yet.