nvidia-riva / nvidia-riva/tutorials
Inference failed when using greedy decoder
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 175
- Forks
- 56
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I'm trying the deploy a NeMo conformer CTC model using Riva. It works well when evaluating with NeMo, but Riva is failing to infer when using this command during riva-build -
riva-build speech_recognition -f /servicemaker-dev/ASR-Model-Language-bn-val-wer-0.132.rmir /servicemaker-dev/ASR-Model-Language-bn-val-wer-0.132.riva \
--name=conformer-ctc-med-voicebook-it1-run3 \
--featurizer.use_utterance_norm_params=False \
--featurizer.precalc_norm_time_steps=0 \
--featurizer.precalc_norm_params=False \
--ms_per_timestep=40 \
--nn.fp16_needs_obey_precision_pass \
--chunk_size=0.16 \
--left_padding_size=1.92 \
--right_padding_size=1.92 \
--decoder_type=greedy \
--language_code=bn-BD --force
Using this command to transcribe an audio file -
/opt/riva/clients/riva_streaming_asr_client --audio_file /audios/a1.wav --language-code bn-BD --riva_uri localhost:50051
And getting this error in the riva speech container -
0105 06:11:48.388618 195 grpc_riva_asr.cc:1464] ASRService.StreamingRecognize called.
I0105 06:11:48.388924 195 grpc_riva_asr.cc:1491] ASRService.StreamingRecognize performing streaming recognition with sequence id: 761739119
I0105 06:11:48.389112 195 grpc_riva_asr.cc:1556] Using model conformer-ctc-med-voicebook-it1-run3 for inference
I0105 06:11:48.389196 195 grpc_riva_asr.cc:1573] Model sample rate= 16000 for inference
I0105 06:11:48.389746 195 riva_asr_stream.cc:214] Detected format: encoding = 1 numchannels = 1 samplerate = 8000 bitspersample = 16
I0105 06:11:48.390592 263 grpc_riva_asr.cc:1227] Creating resampler, audio file sample rate=8000 model sample_rate=16000
E0105 06:11:48.681532 92 ctc-decoder.cc:328] Inference failed in ASR decoder: basic_string::_M_construct null not valid
E0105 06:11:48.681618 92 backend_triton_api.cc:111] Model 'conformer-ctc-med-voicebook-it1-run3-ctc-decoder-cpu-streaming', instance: 'conformer-ctc-med-voicebook-it1-run3-ctc-decoder-cpu-streaming_0': failed executing 1 request(s) as one batch on device 0
W0105 06:13:28.664069 263 grpc_riva_asr.cc:1332] Response timeout. requests sent: 814 received: 52
E0105 06:13:28.664247 195 grpc_riva_asr.cc:1677] ASRService.StreamingRecognize returning failure
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the supplied riva-build configuration and riva_streaming_asr_client command, then inspect the ctc-decoder.cc and grpc_riva_asr.cc locations named in the logs. Compare the greedy-decoder setup with the working NeMo evaluation path. Done means streaming recognition completes without the decoder error or timeout.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100