Decoding speed and accuracy on the transformed onnx model

Hi, thanks for you share of the espnet_onnx system!

I met two problems when I tried to inference thorough your codes. My acoustic model is trained by myself on our own dataset. The AM architecture is the typical Conformer. I downloaded this code on June.

First, the decoding speed is too slow by it. When using torch to decode, the RTF is around 2.32; however it becomes around 20 when using the transformed onnx.

Second, the CER calculated in the torch version is 7.8% while for the onnx, it becomes 10.6%. I think it is probably wrong.

I'm giving some configs here:

export.py

```
import sys
sys.path.append('espnet-master')
sys.path.append('espnet-master/espnet_tts_frontend-master')
sys.path.append('espnet_onnx-master/espnet_onnx/export/asr')
import torch

from export_asr import ModelExport
from espnet2.bin.asr_inference import Speech2Text

if __name__ == '__main__':
m = ModelExport(cache_dir = sys.argv[5])

# export from trained model
speech2text=Speech2Text(
asr_train_config = sys.argv[1],
asr_model_file=sys.argv[2],
lm_train_config=sys.argv[3],
lm_file=sys.argv[4],
)

m.export(model = speech2text, tag_name = 'speech2text', quantize=True)
```

And I get an onnx dir structured like:

asr/onnx/speech2text/
      config.yaml
      feats_stats.npz
      full/
      quantize/

The test wav is a filelist, structured as:
```
bigfar_001_000001 /home/dangfeng/exp_xiandao/for_xiandao/onnx_enh/output_0703/enh/bigfar_001_000001.wav
bigfar_001_000002 /home/dangfeng/exp_xiandao/for_xiandao/onnx_enh/output_0703/enh/bigfar_001_000002.wav
bigfar_001_000003 /home/dangfeng/exp_xiandao/for_xiandao/onnx_enh/output_0703/enh/bigfar_001_000003.wav
bigfar_001_000004 /home/dangfeng/exp_xiandao/for_xiandao/onnx_enh/output_0703/enh/bigfar_001_000004.wav
bigfar_001_000005 /home/dangfeng/exp_xiandao/for_xiandao/onnx_enh/output_0703/enh/bigfar_001_000005.wav
bigfar_001_000006 /home/dangfeng/exp_xiandao/for_xiandao/onnx_enh/output_0703/enh/bigfar_001_000006.wav
...
```

The decoding process is:

decode.py

```
import sys
sys.path.append('espnet_onnx-master/espnet_onnx/asr')

import time
import threading
import librosa
import os
from tqdm import tqdm
from asr_model import Speech2Text

if __name__ == '__main__':
""" step1: load onnx file """
speech2text = Speech2Text(tag_name = 'speech2text', model_dir=sys.argv[3],)

""" step2: ASR """
f = open(sys.argv[1])
lines = f.readlines()
for line in tqdm(lines):
with open(os.path.join(sys.argv[2], 'hyp_flush_1process.trn'),'a') as fout:
wav_name = line.split(' ')[0].strip()
processing_wav = line.split(' ')[1].strip()

start = time.time()
y, sr = librosa.load(processing_wav, sr=16000)
nbest = speech2text(y)
asr_result = nbest[0][0]
end = time.time()

for j in range (len(asr_result)):
fout.write(asr_result[j])
if j != len(asr_result) - 1:
fout.write(' ')
fout.write('\t')
fout.write('(')
fout.write(wav_name)
fout.write('-')
fout.write(wav_name)
fout.write(')')
fout.write('\n')

print('processing: ', processing_wav)
print('Result: ', asr_result)
print('Time: ', end-start, 's')
```

Furthermore, I noticed that you have mentioned there may be some problems for Conformer AM considering ASR in latest issue, has it been fixed?

Looking forward for your reply!