DINO Loss is not decreasing

Hi,

Thank you for sharing the repo.

I am using this repo to pretrain a model with my dataset. Currently, I am using 10k 3D images taken from one organ. The problem that I have is shown in the attached image; DINO loss does not decrease. Here is my config:

dino:
loss_weight: 1.0
head_n_prototypes: 100
head_bottleneck_dim: 256
head_nlayers: 3
head_hidden_dim: 2048
koleo_loss_weight: 0.1
ibot:
loss_weight: 1.0
mask_sample_probability: 0.5
mask_ratio_min_max:
- 0.2
- 0.75
separate_head: true
head_n_prototypes: 100
head_bottleneck_dim: 256
head_nlayers: 3
head_hidden_dim: 2048
train:
batch_size_per_gpu: 16

Did you face similar behavior? Do you have any suggestions?