Allow proper concurrency on different CPU

The functions currently run as async functions, but the pod has N CPU's, and N workers. The API shoudl take full advantage of the N cpu's

When a thread crashes it should also no bring down the API.

Two options, use uvicorn --workers 4 and then re-do QueueManager

Or use from concurrent.future import ProcessPoolExecutor.

There are options

Start by inspecting the API's QueueManager and current async worker entry points. Compare the issue's two proposed approaches—uvicorn workers or concurrent.futures.ProcessPoolExecutor—and determine how worker failures are isolated. Done means the API uses the available CPUs for concurrent jobs without a worker crash bringing down the API.

Written by the indexing model from the issue text.