Unexpected crash when running simple flow problems with cuda enabled.
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 287
- Forks
- 109
- Avg merge
- 4d 41m
- Merged PRs (30d)
- 5
Description
Describe the bug
Runtime crash when running simple cases with cuda compilation.
To Reproduce
- Compile with cuda and try to run:
bin/geosx -i ../inputFiles/singlePhaseFlow/sourceFlux_1d.xml -o output
bin/geosx -i ../inputFiles/singlePhaseFlow/incompressible_1d.xml -o output
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
bin/geosx -i ../inputFiles/singlePhaseFlow/sourceFlux_1d.xml -o output
Num ranks: 1
GEOS version: 1.1.0
- c++ compiler: gcc 11.4.0
- CUDA compiler version: 12.8
- MPI version: Open MPI v4.1.2, package: Debian OpenMPI, ident: 4.1.2, repo rev: v4.1.2, Nov 24, 2021
- HDF5 version: 1.12.1
- Conduit version: 0.9.2
- VTK version: 9.4.2
- RAJA version: 2025.3.0
- umpire version: 2025.3.0
- chai version: 2025.3.0
- adiak version: ..
- caliper version: ..
- METIS version: 5.1.0
- PARAMETIS version: 4.0.0
- superlu_dist version: 6.3.0
- suitesparse version: 5.7.9
- hypre version: v2.33.0-20-g907a2d07b (HEAD)
- trilinos version: 16.1.0
- Python3 version: 3.10.12
- CUDAToolkit version: 12.8.61
Started at 2025-10-03 22:39:25.189185325
Opened XML file: /workspace/GEOS/inputFiles/singlePhaseFlow/sourceFlux_1d.xml
Solvers: adding SinglePhaseFVM SinglePhaseFlow
Mesh: adding InternalMesh mesh1
Geometry: adding Box source
Geometry: adding Box sink
Events: adding PeriodicEvent outputs
Events: adding PeriodicEvent solverApplications
Events: adding PeriodicEvent restarts
FiniteVolume: adding TwoPointFluxApproximation singlePhaseTPFA
FieldSpecifications: adding FieldSpecification Porosity
FieldSpecifications: adding FieldSpecification initialPressure
FieldSpecifications: adding SourceFlux sourceTerm
FieldSpecifications: adding FieldSpecification sinkTerm
Outputs: adding Silo siloOutput
Outputs: adding Restart restartOutput
Constitutive: adding CompressibleSinglePhaseFluid water
Constitutive: adding CompressibleSolidConstantPermeability rock
Constitutive: adding NullModel nullSolid
Constitutive: adding PressurePorosity rockPorosity
Constitutive: adding ConstantPermeability rockPerm
ElementRegions: adding CellElementRegion Region1
####################################################################################################
## Mesh generation ##
####################################################################################################
mesh1: total number of nodes = 44
mesh1: total number of elems = 10
MPI Partitioning information:
--------------------------------------------------------------------------------------------------------------------
| Mesh partitioning over ranks |
|------------------------------------------------------------------------------------------------------------------|
| Ranks | Nodes | Edges | Faces | Elems |
|------------------|-----------------------|-----------------------|-----------------------|-----------------------|
| | Local | Ghost | Total | Local | Ghost | Total | Local | Ghost | Total | Local | Ghost | Total |
|------------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
| 0 | 44 | 0 | 44 | 84 | 0 | 84 | 51 | 0 | 51 | 10 | 0 | 10 |
|------------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
| sum | 44 | 0 | 44 | 84 | 0 | 84 | 51 | 0 | 51 | 10 | 0 | 10 |
| min | 44 | 0 | 44 | 84 | 0 | 84 | 51 | 0 | 51 | 10 | 0 | 10 |
| max | 44 | 0 | 44 | 84 | 0 | 84 | 51 | 0 | 51 | 10 | 0 | 10 |
|------------------|-----------------------|-----------------------|-----------------------|-----------------------|
| min(local/total) | 1 | 1 | 1 | 1 |
| max(local/total) | 1 | 1 | 1 | 1 |
--------------------------------------------------------------------------------------------------------------------
## End of Mesh generation ##
####################################################################################################
####################################################################################################
## Numerical Methods ##
####################################################################################################
meshBodyName/meshLevelName/regionName/subRegionName = mesh1/Level0/Region1/block1, 1 quadrature point
ConstitutiveModels/water allocated
ConstitutiveModels/rock allocated
ConstitutiveModels/nullSolid allocated
ConstitutiveModels/rockPorosity allocated
ConstitutiveModels/rockPerm allocated
## End of Numerical Methods ##
####################################################################################################
####################################################################################################
## Import fields ##
####################################################################################################
mesh1: importing field data from mesh dataset
mesh level = Level0
volumic fields on Region1/block1
mesh level = singlePhaseTPFA
volumic fields on Region1/block1
## End of Import fields ##
####################################################################################################
Time: 0.00e+00 s, dt: 1000 s, Cycle: 0
####################################################################################################
## TIMESTEP ##
####################################################################################################
## - Time : 00h00m00s out of 05h33m20s (0% completed) ##
## 0 s / 20000 s ##
## - Delta Time : 00h16m40s (1000 s) ##
## - Cycle : 0 ##
----------------------------------------------------------
| The summary of declared fields and coupling |
|--------------------------------------------------------|
| | name | comp | N global DOF |
|-----|------------------------|--------|----------------|
| 0 | singlePhaseVariables | 1 | 10 |
----------------------------------------------------------
Received signal 11: Segmentation fault
** StackTrace of 16 frames **
Frame 0: /lib/x86_64-linux-gnu/libc.so.6
Frame 1: hypre_CSRMatrixFnorm
Frame 2: hypre_CSRMatrixTaggedFnorm
Frame 3: hypre_ParCSRMatrixComputeScalingTagged
Frame 4: geos::HypreMatrix::computeScalingVector(geos::HypreVector&) const
Frame 5: geos::PhysicsSolverBase::solveLinearSystem(geos::DofManager const&, geos::HypreMatrix&, geos::HypreVector&, geos::HypreVector&)
Frame 6: geos::PhysicsSolverBase::solveNonlinearSystem(double const&, double const&, int, geos::DomainPartition&)
Frame 7: geos::PhysicsSolverBase::nonlinearImplicitStep(double const&, double const&, int, geos::DomainPartition&)
Frame 8: geos::PhysicsSolverBase::solverStep(double const&, double const&, int, geos::DomainPartition&)
Frame 9: geos::PhysicsSolverBase::execute(double, double, int, int, double, geos::DomainPartition&)
Frame 10: geos::EventBase::execute(double, double, int, int, double, geos::DomainPartition&)
Frame 11: geos::EventManager::run(geos::DomainPartition&)
Frame 12: geos::GeosxState::run()
Frame 13: main
Frame 14: /lib/x86_64-linux-gnu/libc.so.6
Frame 15: __libc_start_main
Frame 16: _start
=====
Platform (please complete the following information):
- Machine: H200
- Compiler: gcc 11.4]
- GEOS Version: latest develop
- Cuda 12.8
Additional context
Add any other context about the problem here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the crash with bin/geosx using inputFiles/singlePhaseFlow/sourceFlux_1d.xml and incompressible_1d.xml in a CUDA build. Start at geos::HypreMatrix::computeScalingVector and PhysicsSolverBase::solveLinearSystem, then compare the hypre CSR stack frames; done means both simple flows run without the segmentation fault.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- hpc
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100