Checkered Predictions (Checkered Artifacts?)

Hi,
I have been modifying the scripts of vanilla SAM, mainly to come up with my own training script.
I was kind of successful in that, and training is happening with loss gradually reducing. But I noticed something, when I save the predictions made by the model in every epoch, I observe that there is a checkered lines all over the predictions.
For eg, in the below image, the left one is prediction from epoch 1 and right image is from epoch 117. I observe that although the grid is fading, but it's clearly visible.

Does anyone know what is causing this? Or is it just because the model is not trained for enough number of epochs?
Well, I'm using just 15 image and mask pairs for training (Using image encoder weights, but training prompt encoder and mask decoder from scratch).

I would be grateful if someone can give me some clue if not a proper solution. Thanks in advance!

No specific files or tests are named. Start by reviewing the modified vanilla SAM training scripts and the path that saves predictions, then compare the image encoder, prompt encoder, and mask decoder setup with the 15 image-mask pairs. Done means identifying whether the checkered artifact comes from training, data, or prediction saving, with evidence from a reproducible run.

Written by the indexing model from the issue text.