Crash with Node.js acceptWaveformAsync (heap-use-after-free)
- Lingua principale
- Jupyter Notebook
- Stelle
- 15.1k
- Fork
- 1.8k
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
I'm facing a server crash which ultimately happens because a Recognizer is free'd while still being in use by a call to `acceptWaveformAsync()`.
The issue happens as follows: My Node.js server processes several transcription requests from remote clients and each one is handled with a Recognizer instance. Each chunk of audio input is then being transcribed with calls to `Recognizer.acceptWaveformAsync()`. When the remote client shuts its connection down (e.g. the remote app is closed), the server calls `Recognizer.free()`.
Judging from the AddressSanitizer stack traces, it seems to me that if an `acceptWaveformAsync()` calls was working when `free()` is called, this *can* trigger an "AddressSanitizer: heap-use-after-free" and a crash. But that's just my feeling, when I look at the stack traces. It might be something completely different that I'm not realizing.
* Is this something that should be handled at the level of vosk-api, i.e. an actual bug, and some code additions are needed here?
* Or is this issue something to be handled and taken care of by the application itself?
A couple of different instances of this issue:
**Crash 1**
```
=================================================================
==228348==ERROR: AddressSanitizer: heap-use-after-free on address 0x61100021c314 at pc 0x7fbead4f8bc2 bp 0x7fbea86843e0 sp 0x7fbea86843d0
READ of size 4 at 0x61100021c314 thread T10
#0 0x7fbead4f8bc1 in kaldi::LatticeIncrementalDecoderTpl > >, kaldi::decoder::BackpointerToken>::ProcessNonemitting(float) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0xa28bc1)
#1 0x7fbead553983 in kaldi::LatticeIncrementalDecoderTpl > >, kaldi::decoder::BackpointerToken>::AdvanceDecoding(kaldi::DecodableInterface*, int) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0xa83983)
#2 0x7fbeace8993c in Recognizer::AcceptWaveform(kaldi::Vector&) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x3b993c)
#3 0x7fbeace89d47 in Recognizer::AcceptWaveform(char const*, int) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x3b9d47)
#4 0x7fbead29fa4c in vosk_recognizer_accept_waveform (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x7cfa4c)
#5 0x7fbeaf495704 in ffi_call_unix64 (/app/node_modules/ffi-napi/build/Release/ffi_bindings.node+0x15704)
#6 0x7fbeaf49425b in ffi_call_int (/app/node_modules/ffi-napi/build/Release/ffi_bindings.node+0x1425b)
#7 0x7fbeaf4894a2 in FFI::FFI::AsyncFFICall(uv_work_s*) (/app/node_modules/ffi-napi/build/Release/ffi_bindings.node+0x94a2)
#8 0x1547a93 in worker ../deps/uv/src/threadpool.c:122
#9 0x7fbec38b2608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477
#10 0x7fbec37d5132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132)
0x61100021c314 is located 84 bytes inside of 256-byte region [0x61100021c2c0,0x61100021c3c0)
freed by thread T0 here:
#0 0x7fbec3d9051f in operator delete(void*) ../../../../src/libsanitizer/asan/asan_new_delete.cc:165
#1 0x7fbead4c729b in kaldi::LatticeIncrementalDecoderTpl > >, kaldi::decoder::BackpointerToken>::~LatticeIncrementalDecoderTpl() (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x9f729b)
#2 0x7fbeace8805c in Recognizer::~Recognizer() (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x3b805c)
#3 0x7fbead29fafa in vosk_recognizer_free (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x7cfafa)
#4 0x7fbeaf495704 in ffi_call_unix64 (/app/node_modules/ffi-napi/build/Release/ffi_bindings.node+0x15704)
#5 0x602001ad80cf ()
previously allocated by thread T10 here:
#0 0x7fbec3d8f587 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:104
#1 0xe4b08a in void std::vector >::_M_realloc_insert(__gnu_cxx::__normal_iterator > >, int const&) (/usr/bin/node+0xe4b08a)
#2 0x7fbead4f8982 in kaldi::LatticeIncrementalDecoderTpl > >, kaldi::decoder::BackpointerToken>::ProcessNonemitting(float) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0xa28982)
#3 0x7fbead553983 in kaldi::LatticeIncrementalDecoderTpl > >, kaldi::decoder::BackpointerToken>::AdvanceDecoding(kaldi::DecodableInterface*, int) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0xa83983)
#4 0x7fbeace8993c in Recognizer::AcceptWaveform(kaldi::Vector&) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x3b993c)
#5 0x7fbeace89d47 in Recognizer::AcceptWaveform(char const*, int) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x3b9d47)
#6 0x7fbead29fa4c in vosk_recognizer_accept_waveform (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x7cfa4c)
#7 0x7fbeaf495704 in ffi_call_unix64 (/app/node_modules/ffi-napi/build/Release/ffi_bindings.node+0x15704)
#8 0x602001b37acf ()
Thread T10 created by T0 here:
#0 0x7fbec3cba815 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cc:208
#1 0x155affb in uv_thread_create_ex ../deps/uv/src/unix/thread.c:259
#2 0x155affb in uv_thread_create ../deps/uv/src/unix/thread.c:213
#3 0x1547e6a in init_threads ../deps/uv/src/threadpool.c:230
#4 0x1547e6a in init_once ../deps/uv/src/threadpool.c:257
#5 0x7fbec38bb4de in __pthread_once_slow /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_once.c:116
SUMMARY: AddressSanitizer: heap-use-after-free (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0xa28bc1) in kaldi::LatticeIncrementalDecoderTpl > >, kaldi::decoder::BackpointerToken>::ProcessNonemitting(float)
Shadow bytes around the buggy address:
0x0c228003b810: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c228003b820: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c228003b830: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c228003b840: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c228003b850: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
=>0x0c228003b860: fd fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c228003b870: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
0x0c228003b880: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c228003b890: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c228003b8a0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x0c228003b8b0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==228348==ABORTING
```
**Crash 2**
```
WARNING (VoskAPI:~HashList():util/hash-list-inl.h:117) Possible memory leak: 1799 != 2048: you might have forgotten to call Delete on some Elems
=================================================================
==217906==ERROR: AddressSanitizer: heap-use-after-free on address 0x619001099728 at pc 0x7f6ee27aa312 bp 0x7f6edd3ffc10 sp 0x7f6edd3ffc00
READ of size 1 at 0x619001099728 thread T10
#0 0x7f6ee27aa311 in kaldi::nnet3::NnetComputer::Run() (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0xfda311)
#1 0x7f6ee2875959 in kaldi::nnet3::DecodableNnetLoopedOnlineBase::AdvanceChunk() (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x10a5959)
#2 0x7f6ee287646a in kaldi::nnet3::DecodableAmNnetLoopedOnline::LogLikelihood(int, int) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x10a646a)
#3 0x7f6ee2211f36 in kaldi::LatticeIncrementalDecoderTpl > >, kaldi::decoder::BackpointerToken>::ProcessEmitting(kaldi::DecodableInterface*) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0xa41f36)
#4 0x7f6ee225397b in kaldi::LatticeIncrementalDecoderTpl > >, kaldi::decoder::BackpointerToken>::AdvanceDecoding(kaldi::DecodableInterface*, int) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0xa8397b)
#5 0x7f6ee1b8993c in Recognizer::AcceptWaveform(kaldi::Vector&) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x3b993c)
#6 0x7f6ee1b89d47 in Recognizer::AcceptWaveform(char const*, int) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x3b9d47)
#7 0x7f6ee1f9fa4c in vosk_recognizer_accept_waveform (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x7cfa4c)
#8 0x7f6ee4682704 in ffi_call_unix64 (/app/node_modules/ffi-napi/build/Release/ffi_bindings.node+0x15704)
#9 0x7f6ee468125b in ffi_call_int (/app/node_modules/ffi-napi/build/Release/ffi_bindings.node+0x1425b)
#10 0x7f6ee46764a2 in FFI::FFI::AsyncFFICall(uv_work_s*) (/app/node_modules/ffi-napi/build/Release/ffi_bindings.node+0x94a2)
#11 0x1547a93 in worker ../deps/uv/src/threadpool.c:122
#12 0x7f6ef8963608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477
#13 0x7f6ef8886132 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f132)
0x619001099728 is located 168 bytes inside of 1056-byte region [0x619001099680,0x619001099aa0)
freed by thread T0 here:
#0 0x7f6ef8e4151f in operator delete(void*) ../../../../src/libsanitizer/asan/asan_new_delete.cc:165
#1 0x7f6ee1b880a2 in Recognizer::~Recognizer() (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x3b80a2)
#2 0x7f6ee1f9fafa in vosk_recognizer_free (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x7cfafa)
#3 0x7f6ee4682704 in ffi_call_unix64 (/app/node_modules/ffi-napi/build/Release/ffi_bindings.node+0x15704)
#4 0x602001bd4aef ()
previously allocated by thread T0 here:
#0 0x7f6ef8e40587 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:104
#1 0x7f6ee1bae738 in Recognizer::Recognizer(Model*, float) (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x3de738)
#2 0x7f6ee1f9f884 in vosk_recognizer_new (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0x7cf884)
#3 0x7f6ee4682704 in ffi_call_unix64 (/app/node_modules/ffi-napi/build/Release/ffi_bindings.node+0x15704)
#4 0x60200198cc4f ()
Thread T10 created by T0 here:
#0 0x7f6ef8d6b815 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cc:208
#1 0x155affb in uv_thread_create_ex ../deps/uv/src/unix/thread.c:259
#2 0x155affb in uv_thread_create ../deps/uv/src/unix/thread.c:213
#3 0x1547e6a in init_threads ../deps/uv/src/threadpool.c:230
#4 0x1547e6a in init_once ../deps/uv/src/threadpool.c:257
#5 0x7f6ef896c4de in __pthread_once_slow /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_once.c:116
SUMMARY: AddressSanitizer: heap-use-after-free (/app/node_modules/vosk/lib/linux-x86_64/libvosk.so+0xfda311) in kaldi::nnet3::NnetComputer::Run()
Shadow bytes around the buggy address:
0x0c328020b290: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328020b2a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
0x0c328020b2b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c328020b2c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c328020b2d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c328020b2e0: fd fd fd fd fd[fd]fd fd fd fd fd fd fd fd fd fd
0x0c328020b2f0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328020b300: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328020b310: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328020b320: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c328020b330: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==217906==ABORTING
```
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Valutazione
Questa issue non è ancora stata valutata.