Beam Search Is very Slow in Transformer

I have been using beam size of 3 and alpha 1.0 for beam search decoding looks like it is very slow .
Greedy search takes around 30-40 seconds for generating a sequence of length 250 words.
but beam search takes around 2 minutes ,

Can you help me improve the inference . i tried quantising the model to 8bits it decreased the size of the model but inference time still remains the same.

Any help is appreciated.

Thanks

Start by reproducing inference with beam size 3, alpha 1.0, and sequence length 250, comparing it with greedy search and the 8-bit quantized model. Done means identifying and addressing the cause of the beam-search slowdown, with measured inference time showing an improvement.

Written by the indexing model from the issue text.