How to improve the class type prediction accuracy of Mask-Rcnn

Hi,

Thank you for this repo. I have an issue, please take a look at the details below.

I am training mask-rcnn model using a custom dataset. The model can predict the objects' location, but the predicted class is wrong more than 40% of the time. How can I improve the class prediction?
Are there any configurations I can tweak so that the training can give more importance to the classification part? Like more weightage to classification loss or something, which can improve the model class prediction? Please advise.

Thanks.