microsoft / microsoft/onnxruntime
How to inference on multi-gpus
Open
@pranavsharma is already working on this.
Since Jul 2, 2021.
- Dominant language
- C++
- Stars
- 21.9k
- Forks
- 4.2k
- Avg merge
- 4d 8h
- Merged PRs (30d)
- 179
Description
I have know how to infenrence on single gpu , Use OrtSessionOptionsAppendExecutionProvider_CUDA(session_options, gpu_id).But when I inference on multi-gpus, it reports some error.
like [E:onnxruntime:OnnxruntimeInferenceEnv, cuda_call.cc:103 CudaCall] CUDA failure 700: an illegal memory access was encountered
or [E:onnxruntime:OnnxruntimeInferenceEnv, cuda_call.cc:103 CudaCall] CUDNN failure 4: CUDNN_STATUS_INTERNAL_ERROR
Please tell how to do that
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.