MeasureTransport / MeasureTransport/MParT
Problem with using python MParT with GPU when GPU is being used
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 18
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Description
Someone else was using a GPU as well and so I got the error
>>> import mpart as mt
RuntimeError: cudaDeviceSynchronize() error( cudaErrorDevicesUnavailable): CUDA-capable device(s) is/are busy or unavailable /home/dannys4/git-repos/kokkos/core/src/Cuda/Kokkos_Cuda_Instance.cpp:151
despite there being multiple GPUs (one being unused). I had no luck trying to change the device via os.environ['DEVICE_ID'] = '1'. I will try changing the arguments within the startup file but, regardless, we should
- Create a way so that MParT.py isn't initialized automatically (same for other bindings)
- Document how to initialize it manually
- Document what you can use to change the initialization
- See why we aren't allowed to run Kokkos on GPU while something else is running
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the automatic initialization of MParT.py and the other bindings, then inspect the referenced Kokkos_Cuda_Instance.cpp path and the startup-file arguments. Reproduce the failure with multiple GPUs in use, determine how device selection and concurrent Kokkos use are configured, and document manual initialization and supported options. Done means initialization is controllable and the GPU concurrency behavior is explained.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- hpc
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100