How to set-up a Vosk multi-threads server architecture in NodeJs

Hi Nicolay,

That's not a real issue, just two questions/ a brainstorming/suggestion request, about a server architecture in nodejs.

I'm trying to extend my project [voskJs](https://github.com/solyarisoftware/voskJs) implementing a nodejs server side architecture to manage multiple concurrent Vosk transcript requests.

Here https://github.com/alphacep/vosk-api/issues/498 you told me that the transcript function run on a single core and you rightly suggested to implement a multithread server. So I'm trying to understand how can I use nodejs [worker threads](https://nodejs.org/api/worker_threads.html).

For a server that by example has to manage a single language (consequently say a single model), my idea was
- to init the model once at start-up time (in the main/parent server thread) and afterward
- to run child threads for run-time transcripts (Recognizer), spawning a thread for each request.

But I have a problem: in nodejs working threads in theory can NOT share an object containing functions. See:

1. https://nodejs.org/api/worker_threads.html#worker_threads_worker_workerdata
2. https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm
3. https://stackoverflow.com/a/12303840/1786393

whereas the Vosk Model Object contains functions:

```Javascript
Model {
handle: Buffer(0) [Uint8Array] [
type: {
size: 0,
indirection: 1,
get: [Function: get],
set: [Function: set],
name: 'void'
}
]
}
```

So I fair I can't pass to the Model each thread. I'll verify asap in practice.

Now I have a serious problem because the Vosk Model requires a huge amount of RAM.

By example using English language large model `vosk-model-en-us-aspire-0.2`, it seems to me that Vosk occupy something like ~3 GB RAM (see below the `Maximum resident set size (kbytes): 3253024` line when running `/usr/bin/time --verbose node voskjs --audio=audio/2830-3980-0043.wav --model=models/vosk-model-en-us-aspire-0.2`).

See stdout when running Vosk transcript in single process/request (using voskJs wrapper):

```
$ /usr/bin/time --verbose node voskjs --audio=audio/2830-3980-0043.wav --model=models/vosk-model-en-us-aspire-0.2

log level : 0

LOG (VoskAPI:ReadDataFiles():model.cc:194) Decoding params beam=13 max-active=7000 lattice-beam=6
LOG (VoskAPI:ReadDataFiles():model.cc:197) Silence phones 1:2:3:4:5:6:7:8:9:10:11:12:13:14:15
LOG (VoskAPI:RemoveOrphanNodes():nnet-nnet.cc:948) Removed 1 orphan nodes.
LOG (VoskAPI:RemoveOrphanComponents():nnet-nnet.cc:847) Removing 2 orphan components.
LOG (VoskAPI:Collapse():nnet-utils.cc:1488) Added 1 components, removed 2
LOG (VoskAPI:CompileLooped():nnet-compile-looped.cc:345) Spent 0.00862885 seconds in looped compilation.
LOG (VoskAPI:ReadDataFiles():model.cc:221) Loading i-vector extractor from models/vosk-model-en-us-aspire-0.2/ivector/final.ie
LOG (VoskAPI:ComputeDerivedVars():ivector-extractor.cc:183) Computing derived variables for iVector extractor
LOG (VoskAPI:ComputeDerivedVars():ivector-extractor.cc:204) Done.
LOG (VoskAPI:ReadDataFiles():model.cc:246) Loading HCLG from models/vosk-model-en-us-aspire-0.2/graph/HCLG.fst
LOG (VoskAPI:ReadDataFiles():model.cc:265) Loading words from models/vosk-model-en-us-aspire-0.2/graph/words.txt
LOG (VoskAPI:ReadDataFiles():model.cc:273) Loading winfo models/vosk-model-en-us-aspire-0.2/graph/phones/word_boundary.int
LOG (VoskAPI:ReadDataFiles():model.cc:281) Loading CARPA model from models/vosk-model-en-us-aspire-0.2/rescore/G.carpa

init elapsed : 22131ms
Model {
handle: Buffer(0) [Uint8Array] [
type: {
size: 0,
indirection: 1,
get: [Function: get],
set: [Function: set],
name: 'void'
}
]
}
transcript elapsed : 773ms

{
result: [
{ conf: 0.980891, end: 1.02, start: 0.33, word: 'experience' },
{ conf: 1, end: 1.349903, start: 1.02, word: 'proves' },
{ conf: 0.996779, end: 1.71, start: 1.35, word: 'this' }
],
text: 'experience proves this'
}

Command being timed: "node voskjs --audio=audio/2830-3980-0043.wav --model=models/vosk-model-en-us-aspire-0.2"
User time (seconds): 2.47
System time (seconds): 5.08
Percent of CPU this job got: 31%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:24.23
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 3253024
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 9
Minor (reclaiming a frame) page faults: 808075
Voluntary context switches: 6226
Involuntary context switches: 897
Swaps: 0
File system inputs: 3407640
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0

```
---

Questions:

1. May you confirm that Vosk model RAM usage is ~3 GB RAM (for the mentioned language model)?

2. Using processes instead of threads:
If I can't user worker threads, reusing a shared memory for the huge Model object, the alternative could be to implement a multi-process architecture of workers, but in this case any worker process must load the model separately (e.g. > ~3 GB). So I have an 8 cores host, and I foresee say 7 child/worker processes, the total amount of RAM in the host must me something > ~3GB * 7 = >~21 GB! That's insane. Any suggestion for an alternative solution (in nodejs)?

3. Using vosk-server
I guess at the end of the day a nodejs server could just do some IPC with the [Vosk-Server](https://github.com/alphacep/vosk-server) you implemented. How much RAM and cpu cores vosk-server requires?

Thanks for your patience
Giorgio