Allow multi-thread streaming using the same weight loaded once in memory

I'm trying out this project for a live transcription usecase in a call. We use vosk at the moment (in https://github.com/nextcloud/live_transcription/) which supports loading the model weights once in the memory and then starting new threads with light-er recognize objects reading the same weights to process the output but keeping their own state and cache.
This is not possible at this moment due to a mutex https://github.com/mudler/parakeet.cpp/blob/e75de9b6b9b688fd293aa22f7e27aa724ea286f8/src/ggml_graph.cpp#L28-L35
One process can process only one stream at a time.

I'm not familiar with the code so did an experiment and asked AI if there is a possibility to work like llama.cpp here which uses slots to entertain parallel requests using the same loaded weights, and works with the same underlying ggml library.
It has successfully changed the code to make it possible for multiple threads to transcribe at the same time, in the same process by using a new Backend in each thread as opposed to one shared Backend + mutex guard.
The tests ran on an AMD CPU but theoritically should not cause issues with GPU systems.

Below are some reproduction steps and the patch:

parakeet-thread-local-backend.patch
concurrent_streams.py

python concurrent_streams.py --lib ./libparakeet.so --model ./nemotron-3.5-asr-streaming-0.6b-q8_0.gguf --threads 4 ./audio/en_9min_16k* --seconds 90

Start by reviewing the mutex in src/ggml_graph.cpp at lines 28-35, then inspect parakeet-thread-local-backend.patch and concurrent_streams.py. Run the provided Python command with four threads and the supplied model and audio files. Done means multiple streams can be transcribed concurrently in one process while sharing weights, including on GPU systems.