EnzymeAD / EnzymeAD/ReactantServer.jl
KServe V2 Inference Streaming
Open
enhancement
- Dominant language
- Julia
- Stars
- 5
- Forks
- 2
- Avg merge
- 15m
- Merged PRs (30d)
- 10
Description
Currently we only allow unary RPC due to some issues with the unpublished gRPCServer.jl package when it comes to cancellation. We should add support for streaming RPC to reduce RPC overhead for workflows where this is needed.
Contributor guide
Assessment
This issue has not been assessed yet.