python: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 0: invalid start byte
- Lenguaje dominante
- Jupyter Notebook
- Estrellas
- 15.1k
- Forks
- 1.8k
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
So far I've not been able to reproduce this problem, but while using [nerd-dictation](https://github.com/ideasman42/nerd-dictation), we have hit [a Vosk decoding issue](https://github.com/ideasman42/nerd-dictation/pull/91) that appears to be rooted in the Bosk Python API code. I am running Python version 3.6 on CentOS 7 (which gets updates form Red Hat until 2024) while using the `vosk-model-en-us-0.42-gigaspeech` model.
You can see the backtrace below. Notice that the last line triggers an error within the Vosk API at ["vosk/__init__.py", line 194, in FinalResult](https://github.com/alphacep/vosk-api/blob/master/python/vosk/__init__.py#L194)
```py
Traceback (most recent call last):
File "./nerd-dictation", line 1962, in
main()
File "./nerd-dictation", line 1958, in main
args.func(args)
File "./nerd-dictation", line 1845, in
vosk_grammar_file=args.vosk_grammar_file,
File "./nerd-dictation", line 1440, in main_begin
vosk_grammar_file=vosk_grammar_file,
File "./nerd-dictation", line 1215, in text_from_vosk_pipe
json_text = rec_handle_fn_wrapper_from_final_result()
File "./nerd-dictation", line 1054, in rec_handle_fn_wrapper_from_final_result
json_text = rec.FinalResult()
File "/usr/src/nerd-dictation/lib64/python3.6/site-packages/vosk/__init__.py", line 194, in FinalResult
return _ffi.string(_c.vosk_recognizer_final_result(self._handle)).decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 0: invalid start byte
```
@ideasman42, the developer of [nerd-dictation](https://github.com/ideasman42/nerd-dictation) suggests that [this could be fixed](https://github.com/ideasman42/nerd-dictation/pull/91#issuecomment-1441027751) in Vosk by adding `errors=ignore`. For example:
```py
>>> b'A\xaeB'.decode('utf-8', errors='ignore')
'AB'
```
There are 4 different locations where text is decoded to UTF-8, so perhaps they need fixed up as well:
1. https://github.com/alphacep/vosk-api/blob/master/python/vosk/__init__.py#L188
2. https://github.com/alphacep/vosk-api/blob/master/python/vosk/__init__.py#L191
3. https://github.com/alphacep/vosk-api/blob/master/python/vosk/__init__.py#L194
4. https://github.com/alphacep/vosk-api/blob/master/python/vosk/__init__.py#L267
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.