Unexpected inference results from Flan-T5 XXL converted to ctranslate2 with version 4.2.1 and 4.1.1 (using tensor parallel)

I'm using the Flan-t5 XXL of the shelf model in our project and for deployment we have converted it to ctranslate2 version using following command
ct2-transformers-converter --model ~/input_folder/ --output_dir ~/flant5_ct2/

Now I'm hosting the model as gRPC server, while starting under tensor parallel mode like
ctranslate2.Translator(checkpoint_path, device="cuda", tensor_parallel=True)

I started the server with mpirun with 2 instances to allow tensor parallel to kick in. This works well and model is loaded evenly across to 2 GPUs
mpirun -n 2 python model_server.py

Now when I run inference on it, it returns following result as response to my prompt ("Who is president of united states?")
"< pad >< pad >< pad >< pad >< pad >< pad >"

Now this is a strange behaviour which happens only with ctranslate2==4.2.1

Some suggestions to fix it would really helpful here.

Start by reproducing the issue with the ct2-transformers-converter command, model_server.py, and mpirun -n 2 using Flan-T5 XXL. Compare inference output between ctranslate2 4.1.1 and 4.2.1 with tensor parallel enabled. Done means the deployment returns the expected response instead of repeated <pad> tokens.