WARNING :root:NaN or Inf in input tensor problem

INFO:global:Progress: 110 / 83320 [0%], Speed: 1.173 s/iter, ETA 1:03:07 (D:H:M)

WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
[2020-08-02 19:30:50,046-rk0-train_siammask_refine.py#290] Epoch: [1][120/4166] lr: 0.010000 batch_time: 0.435500 (1.143975) data_time: 0.000047 (0.692028) rpn_cls_loss: 0.226122 (0.179825) rpn_loc_loss: 0.189315 (0.190289) rpn_mask_loss: inf (inf) siammask_loss: inf (inf) mask_iou_mean: 0.000000 (0.000000) mask_iou_at_5: 0.000000 (0.000000) mask_iou_at_7: 0.000000 (0.000000)
INFO:global:Epoch: [1][120/4166] lr: 0.010000 batch_time: 0.435500 (1.143975)data_time: 0.000047 (0.692028) rpn_cls_loss: 0.226122 (0.179825) rpn_loc_loss: 0.189315 (0.190289) rpn_mask_loss: inf (inf) siammask_loss: inf (inf) mask_iou_mean: 0.000000 (0.000000) mask_iou_at_5: 0.000000 (0.000000) mask_iou_at_7: 0.000000 (0.000000)
[2020-08-02 19:30:50,046-rk0-log_helper.py# 97] Progress: 120 / 83320 [0%], Speed: 1.144 s/iter, ETA 1:02:26 (D:H:M)

INFO:global:Progress: 120 / 83320 [0%], Speed: 1.144 s/iter, ETA 1:02:26 (D:H:M)

WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
WARNING:root:NaN or Inf found in input tensor.
[2020-08-02 19:31:00,345-rk0-train_siammask_refine.py#290] Epoch: [1][130/4166] lr: 0.010000 batch_time: 0.420345 (1.135102) data_time: 0.000030 (0.685210) rpn_cls_loss: 0.094134 (0.179464) rpn_loc_loss: 0.174147 (0.190233) rpn_mask_loss: inf (inf) siammask_loss: inf (inf) mask_iou_mean: 0.000000 (0.000000) mask_iou_at_5: 0.000000 (0.000000) mask_iou_at_7: 0.000000 (0.000000)

When I train the refine model, the above problem occurs.Then when I adjust lr from 0.01 to 0.001，a warning still appears after training the third epoch.This is my loss. Can anyone help me solve this problem?Thanks！
PICTURE:/home/wzw/.config/tencent-qq//AppData/file//sendpix4.jpg

Start by reproducing refine-model training with the reported learning rates and inspect train_siammask_refine.py around the logged loss values. Review log_helper.py where the NaN or Inf warnings are emitted, then compare the training output with the issue's logs. Done means the refine training no longer produces infinite losses or repeated NaN/Inf warnings.

Written by the indexing model from the issue text.