Model Overfitting

Hi Adrian,
First of all, Thanks for this great work.
I was following the PyTorch version of this code for the training of the FAN model on my own dataset. I have used around 26K images for training and 3K for validation. Also, the test set is separated from my own dataset and that is around 7K images other than training and validation. The images have different variations wrt poses, lighting, and other necessary things, and overall the dataset prepared is very good. The issue I am facing is model is performing well on all these images i.e. training, validation, and testing while when I visualize the model's results on some other images they are poor (points are distorted or they shrink around the face). I am not able to find out the issue, why is it happening so?
Can you help me with this or point out any possible issue on which I can work to resolve this problem.