[Feature Request]: Add optional thread safety to RunInference
Open
ml
new feature
P2
python
run-inference
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What would you like to happen?
There may be models which are not thread safe and will run into issues using RunInference (which relies on shared.py under the covers). We should add the ability to force type safety via locking when performing inference.
### Issue Priority
Priority: 2
### Issue Component
Component: run-inference
Contributor guide
Assessment
This issue has not been assessed yet.