How to finetune on new dataset

I have collected 10k identical subject IDs from internet. I combined the collected data with emore dataset (85742 subject IDs). So, I have total of subject ID is 86742 subject IDs. I want to use my pretrained model (trained from emore dataset with setting ` config.ckpt_embedding = False` for recognition task.

https://github.com/deepinsight/insightface/blob/3866cd77a6896c934b51ed39e9651b791d78bb57/recognition/sample_config.py#L10

Should I retrain the network from scratch or finetune it with pretrained model from Emore dataset? If I finetune on Emore dataset, which layers should I finetune, i.e fc1, and fc7?