Whisper outputs repeating same and consecutive tokens

System Info

This isn't an illusion, because the same audio file is normal in faster whisper, this is a Chinese audio file, in which

faster whisper would output

這個不是現在就是已經開始在跑了那你看你可不可以你好你看有沒有他混在一起中宣好大配你出國可以耶配你出國你看是不是可以他三支你的聲音他就很難聽到嘛對啊所以是換麥克風就好還是

TensorRT-LLM would output

這個不是現在就是已經開始在跑了那你看你可不可以你好你看有沒有喔他混在一起中宣好大配你出國欸可以耶配你出國你看是不是可以他三支你的聲音他就很難聽到嘛對啊所以是換麥克風就好還是欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸

TensorRT-LLM would repeat 欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸欸

all the settings follow https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/whisper, except i set the language token to zh and beam size to 1

Here are faster whisper settings, align to beam size to 1 as well.

from faster_whisper import WhisperModel
import librosa
import time

model_size = "large-v2"

# Run on GPU with FP16
model = WhisperModel(model_size, device="cuda", compute_type="int8_float16")

segments, info = model.transcribe(
    "/home/alec.tu/output_25s.wav",
    beam_size=1,
    without_timestamps=True,
    condition_on_previous_text=False,
)

for segment in segments:
    print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

Setting beam size to 2 or 5 would solve the problem, However, Faster Whisper doesn't require increasing the beam size.

I found out some related settings are used in faster whisper, Does TensorRT-LLM also support these settings?

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

download the audio file, and run the whisper example, setting beam size to 1 and language token to zh

Expected behavior

The output should be closed to faster whisper

actual behavior

tensorrt-LLM outputs repeating same and consecutive tokens, where faster whisper did not.

additional notes

TensorRT-LLM version is v0.18.1

Start with the Whisper example under examples/whisper and reproduce the reported Chinese-audio case using beam size 1 and the zh language token. Compare the available decoding settings with faster_whisper/transcribe.py and determine which settings TensorRT-LLM supports. Done means the repeated-token behavior is resolved or the supported setting and limitation are documented.