I tried madlad400, but there is a problem with the output if it is float16

Hi.

I tried madlad400, but there is a problem with the output if it is float16

$ python convert.py --model google/madlad400-3b-mt
$ python t5.py --model google/madlad400-3b-mt --prompt "<2ja>A tasty apple"

[INFO] Generating with T5...
Input:  <2ja>A tasty apple
リンゴの味
Time: 20.28 seconds, tokens/s: 0.30

$ python convert.py --model google/madlad400-3b-mt --dtype float16
$ python t5.py --model google/madlad400-3b-mt --dtype float16 --prompt "<2ja>A tasty apple"

[INFO] Generating with T5...
Input:  <2ja>A tasty apple
<unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk><unk>
Time: 25.66 seconds, tokens/s: 3.90

Thank.

Reproduce the reported behavior with convert.py and t5.py using google/madlad400-3b-mt, both with default precision and --dtype float16. Compare the generated output and trace the float16 path in those entry points; done means float16 no longer produces repeated tokens for the shown prompt.