Is it necessary to retrain the MVIT model when modifying the PATCH_KERNEL parameter during inference?

Hello,

To what extent does modifying the PATCH_KERNEL size during MVIT inference impact the need for retraining the model, considering factors like the magnitude of change and downstream task performance?

For a given change in PATCH_KERNEL size (e.g., 10% increase), can MVIT still produce accurate results without retraining, or is retraining essential for maintaining performance?