Performance tips

Hi, **this isn't really an issue**, but I'm running Vosk on an Anki Vector robot - it has a Qualcomm APQ8009 CPU, which has 4 Cortex-A7 cores. It runs embedded Linux built with Yocto.

After some tuning (mainly just limiting the grammar), it actually runs at an acceptable speed, however I'd like to **squeeze as much performance as I can. I am wondering if you have any tips.**

The only real difference between it and a Pi 2 is a strictly-softfp environment, in case that affects anything. That isn't due to any limitations of the CPU, just has to do with proprietary Qualcomm blobs needing to run in the same environment. From what I can tell, this doesn't prevent it from actually using an FPU, it just uses integer registers rather than fp registers.

I am essentially using your Android build script, except modified to use my own (GCC 10) toolchain built with crosstool-ng.

Here are my current findings:

- `neon` and `neon-vfpv4` don't seem to cause a difference in performance
- `USE_THREAD=1 NUM_THREADS=4` actually compiles and runs, but makes the performance *worse*
- The en-US zamia model runs considerably faster than the regular small en-US model
- probably not as accurate, but it's accurate enough for my application
- SetEndpointerDelays, SetWords, SetPartialWords don't really seem to do anything
- I am using my own webrtcvad-based VAD implementation, and am just feeding stuff to Vosk then getting a FinalResult once my VAD has detected the end of speech. That's probably why EndpointerDelays does nothing.

This is my current Go code, in the case you see something which could be tuned:

```
func InitVosk() {
loadIntents()
var err error
model, err = vosk.NewModel("/anki/data/assets/cozmo_resources/cloudless/en-US/model")
if err != nil {
log.Fatal("model not found", err)
}
rec, err = vosk.NewRecognizerGrm(model, 16000, GetGrammerList("en-US"))
if err != nil {
log.Fatal("error making rec:", err)
}
// does this actually do anything
rec.SetMaxAlternatives(0)
rec.SetEndpointerDelays(3, 0, 0)
}

func Process(chunk []byte) string {
if len(chunk) == 0 {
fmt.Println("empty chunk")
return ""
}
// todo: experiment with giving acceptwaveform smaller or bigger chunks
stop, _ := DetectEndOfSpeech(chunk)
rec.AcceptWaveform(chunk)
if stop {
var jres map[string]interface{}
json.Unmarshal([]byte(rec.FinalResult()), &jres)
transcribedText := jres["text"].(string)
fmt.Println("transcribed text: " + transcribedText)
return transcribedText
}
return ""
}
```