How to get the fine-tune baseline results?

Hi,
how to fine-tune the model from the third-view dataset on the first-view dataset?

I tried to directly fine-tune the trained model from https://github.com/gsig/charades-algorithms by using this script https://github.com/gsig/actor-observer/blob/master/exp/baseline_resnet152imagenet.py.

However, the results is only 22, which is the same with the model fine-tuned from ImageNet, which indicates the third-view pre-training has no effect on the first-view performance.
Could you please indicate how to correctly fine-tune the model trained from the third-view dataset?

Thanks.