nvidia-riva / nvidia-riva/tutorials

Inference failed when using greedy decoder

Open
#112 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
175
Forks
56
PR merge metrics
No merged PRs in 30d

Description

Hi, I'm trying the deploy a NeMo conformer CTC model using Riva. It works well when evaluating with NeMo, but Riva is failing to infer when using this command during riva-build -

riva-build speech_recognition -f /servicemaker-dev/ASR-Model-Language-bn-val-wer-0.132.rmir /servicemaker-dev/ASR-Model-Language-bn-val-wer-0.132.riva \
--name=conformer-ctc-med-voicebook-it1-run3 \
--featurizer.use_utterance_norm_params=False \
--featurizer.precalc_norm_time_steps=0 \
--featurizer.precalc_norm_params=False \
--ms_per_timestep=40 \
--nn.fp16_needs_obey_precision_pass \
--chunk_size=0.16 \
--left_padding_size=1.92 \
--right_padding_size=1.92 \
--decoder_type=greedy \
--language_code=bn-BD --force

Using this command to transcribe an audio file -
/opt/riva/clients/riva_streaming_asr_client --audio_file /audios/a1.wav --language-code bn-BD --riva_uri localhost:50051
And getting this error in the riva speech container -

0105 06:11:48.388618   195 grpc_riva_asr.cc:1464] ASRService.StreamingRecognize called.
I0105 06:11:48.388924   195 grpc_riva_asr.cc:1491] ASRService.StreamingRecognize performing streaming recognition with sequence id: 761739119
I0105 06:11:48.389112   195 grpc_riva_asr.cc:1556] Using model conformer-ctc-med-voicebook-it1-run3 for inference
I0105 06:11:48.389196   195 grpc_riva_asr.cc:1573] Model sample rate= 16000 for inference
I0105 06:11:48.389746   195 riva_asr_stream.cc:214] Detected format: encoding = 1 numchannels = 1 samplerate = 8000 bitspersample = 16
I0105 06:11:48.390592   263 grpc_riva_asr.cc:1227] Creating resampler, audio file sample rate=8000 model sample_rate=16000
E0105 06:11:48.681532 92 ctc-decoder.cc:328] Inference failed in ASR decoder: basic_string::_M_construct null not valid
E0105 06:11:48.681618 92 backend_triton_api.cc:111] Model 'conformer-ctc-med-voicebook-it1-run3-ctc-decoder-cpu-streaming', instance: 'conformer-ctc-med-voicebook-it1-run3-ctc-decoder-cpu-streaming_0': failed executing 1 request(s) as one batch on device 0
W0105 06:13:28.664069   263 grpc_riva_asr.cc:1332] Response timeout. requests sent: 814 received: 52
E0105 06:13:28.664247   195 grpc_riva_asr.cc:1677] ASRService.StreamingRecognize returning failure

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the supplied riva-build configuration and riva_streaming_asr_client command, then inspect the ctc-decoder.cc and grpc_riva_asr.cc locations named in the logs. Compare the greedy-decoder setup with the working NeMo evaluation path. Done means streaming recognition completes without the decoder error or timeout.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.