OpenFOAM2022+AmgX+PETSc-after determining tmp storage requirements for exclusive_scan: cudaErrorInvalidDevice: invalid device ordinal
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 692
- Forks
- 197
- PR merge metrics
- No merged PRs in 30d
Description
After initial hiccups with building components, I am now facing problems with running icoFoam (OpenFOAM)
with execution dumping core with the following Error message.I am running this on Laptop having GEForce RTX 3060 - GPU memory is 6GB - Appreciate your help
Number of GPU devices :: 1
AMGX version 2.2.0.132-opensource
Built on Aug 4 2022, 23:33:25
Compiled with CUDA Runtime 11.7, using CUDA driver 11.7
Cannot read file as JSON object, trying as AMGX config
Converting config string to current config version
Parsing configuration string: exception_handling=1 ;
Initializing AmgX-p
Initializing AmgX Linear Solver p
terminate called after throwing an instance of 'thrust::system::system_error'
what(): after determining tmp storage requirements for exclusive_scan: cudaErrorInvalidDevice: invalid device ordinal
Caught signal 6 - SIGABRT (abort)
/usr/lib/openfoam/amgx/build/libamgxsh.so : ()+0xf8864c
/lib/x86_64-linux-gnu/libpthread.so.0 : ()+0x14420
/lib/x86_64-linux-gnu/libc.so.6 : gsignal()+0xcb
/lib/x86_64-linux-gnu/libc.so.6 : abort()+0x12b
/lib/x86_64-linux-gnu/libstdc++.so.6 : ()+0x9e911
/lib/x86_64-linux-gnu/libstdc++.so.6 : ()+0xaa38c
/lib/x86_64-linux-gnu/libstdc++.so.6 : ()+0xaa3f7
/lib/x86_64-linux-gnu/libstdc++.so.6 : ()+0xaa6a9
/home/sambath/OpenFOAM/sambath-v2112/platforms/linux64GccDPInt32Opt/lib/libfoam2csr.so : int* thrust::cuda_cub::detail::exclusive_scan_n_impl<thrust::cuda_cub::par_t, int*, long, int*, int, thrust::plus >(thrust::cuda_cub::execution_policythrust::cuda_cub::par_t&, int*, long, int*, int, thrust::plus)+0x32a
/home/sambath/OpenFOAM/sambath-v2112/platforms/linux64GccDPInt32Opt/lib/libfoam2csr.so : AmgXCSRMatrix::setValuesLDU(int, int, int, int, int, int const*, int const*, int, int const*, int const*, double const*, double const*, double const*, double const*)+0x9c1
/home/sambath/OpenFOAM/sambath-v2112/platforms/linux64GccDPInt32Opt/lib/libpetscFoam.so : Foam::amgxSolver::offloadMatrixArrays(AmgXCSRMatrix&, int&, int&, int&) const+0x761
/home/sambath/OpenFOAM/sambath-v2112/platforms/linux64GccDPInt32Opt/lib/libpetscFoam.so : Foam::amgxSolver::scalarSolve(Foam::Field&, Foam::Field const&, unsigned char) const+0x687
/home/sambath/OpenFOAM/sambath-v2112/platforms/linux64GccDPInt32Opt/lib/libpetscFoam.so : Foam::amgxSolver::solve(Foam::Field&, Foam::Field const&, unsigned char) const+0x48
/usr/lib/openfoam/openfoam2112/platforms/linux64GccDPInt32Opt/lib/libfiniteVolume.so : Foam::fvMatrix::solveSegregated(Foam::dictionary const&)+0x625
/usr/lib/openfoam/openfoam2112/platforms/linux64GccDPInt32Opt/lib/libfiniteVolume.so : Foam::fvMatrix::solveSegregatedOrCoupled(Foam::dictionary const&)+0x498
/usr/lib/openfoam/openfoam2112/platforms/linux64GccDPInt32Opt/lib/libfiniteVolume.so : Foam::fvMesh::solve(Foam::fvMatrix&, Foam::dictionary const&) const+0x28
icoFoam : ()+0x289e9
/lib/x86_64-linux-gnu/libc.so.6 : __libc_start_main()+0xf3
icoFoam : ()+0x2a28e
Aborted (core dumped)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing icoFoam with the reported OpenFOAM, AmgX, PETSc, CUDA, and RTX 3060 setup. Trace the failure from AmgXCSRMatrix::setValuesLDU and Foam::amgxSolver::offloadMatrixArrays, then determine why the CUDA device ordinal is invalid; done means identifying a confirmed cause and a reproducible resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- hpc
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100