Detection tutorial failed to converge on another dataset

Hi,
I used another dataset with spacing [0.18, 0.18, 0.18], I changed the intensity_transform parameters, the config parameters(including anchors) and made my dataset_fold[i].json accordingly, I followed all the instructions for training LUNA16 including the same resampling step. In addition, I added some codes to visualize the patch right before they entered the neural-network.
I ran the training script provided in the tutorial both on LUNA16 and the other dataset. The LUNA16 did converge well but the other dataset failed to converge. And by checking if those patches match the GT boxes using the code mentioned before, both datasets looks fine right before they enter the network.
The config_train.json file I used to run the other datset is provided below, most of the sizes of GT boxes are around(3mm, 3mm, 3mm), since I didn't find any notes about the unit of the "base_anchor_shapes" I also tried [[2, 2, 2], [4, 4, 4], [8, 8, 8]], but neither one worked unfortunately :
{
"gt_box_mode": "cccwhd",
"lr": 1e-2,
"spacing": [0.18, 0.18, 0.18],
"batch_size": 4,
"patch_size": [128,128,128],
"val_patch_size": [256,256,256],
"fg_labels": [0],
"n_input_channels": 1,
"spatial_dims": 3,
"score_thresh": 0.02,
"nms_thresh": 0.22,
"returned_layers": [1,2],
"conv1_t_stride": [2,2,2],
"base_anchor_shapes": [[16,16,16],[45,45,45],[100,100,100]],
"balanced_sampler_pos_fraction": 0.3
}
Shortcut of results on the other dataset:
144/150, train_loss: 0.6934
145/150, train_loss: 0.4031
146/150, train_loss: 0.4272
147/150, train_loss: 0.5918
148/150, train_loss: 0.5977
149/150, train_loss: 0.3677
150/150, train_loss: 0.3833
Training time: 405.1036174297333s
epoch 20 average loss: 0.5507
saved last model
Validation time: 531.1751585006714s
2022-11-10 17:02:25,108 - Start COCO metric computation...
2022-11-10 17:02:25,122 - Statistics for COCO metrics finished (t=0.01s).
2022-11-10 17:02:25,122 - COCO metrics computed in t=0.01s.
{'mAP_IoU_0.10_0.50_0.05_MaxDet_100': 0.0, 'nodule_mAP_IoU_0.10_0.50_0.05_MaxDet_100': 0.0, 'AP_IoU_0.10_MaxDet_100': 0.0, 'nodule_AP_IoU_0.10_MaxDet_100': 0.0, 'mAR_IoU_0.10_0.50_0.05_MaxDet_100': 0.0, 'nodule_mAR_IoU_0.10_0.50_0.05_MaxDet_100': 0.0, 'AR_IoU_0.10_MaxDet_100': 0.0, 'nodule_AR_IoU_0.10_MaxDet_100': 0.0}
current epoch: 20 current metric: 0.0000 best metric: 0.0108 at epoch 5