PBC GDF/CDERI corruption at CPU-GPU mapped-memory boundary on AMD CPU + NVIDIA GPU systems
Open
@sunqm is already working on this.
Since May 10, 2026.
- Dominant language
- Cuda
- Stars
- 351
- Forks
- 84
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 35
Description
## Description
I encountered this issue when running `UKS.py` for a defect-system calculation. With the same input settings, the first SCF step already gives different results on different machines.
### Wrong result
```text
E1 = -1886.2263122955874 Ecoul = 498.7505538078927 Exc = -362.2990524302787
init E= -2704.40328934829
cond(S) = 773470.6758133224
Correct result
E1 = -1886.2263122955874 Ecoul = 534.0687392940991 Exc = -364.5454736907426
init E= -2671.33152512255
cond(S) = 773470.6758133224
This bug later causes the gradients to blow up during geometry optimization.
Suspected Location
I traced the problem to the CPU-GPU boundary in the PBC density fitting code. In particular, the issue appears to be related to reading from or writing to CUDA mapped host memory.
Bug 1: gpu4pyscf/pbc/df/rsdf_builder.py
The original code uses:
libpbc.store_col_segment(
cderi.ctypes,
ctypes.cast(j3c.data.ptr, ctypes.c_void_p),
ctypes.c_int(naux), ctypes.c_int(nao_pairs),
ctypes.c_int(p0), ctypes.c_int(p1))
Changing it to a CPU-mediated copy fixes this part:
cderi[:, p0:p1] = j3c.get()
Bug 2: gpu4pyscf/pbc/df/df.py
The original code uses:
out = asarray(cderi_sparse[p0:p1])
Changing it to a safe host copy before transferring to GPU fixes this part:
out = cp.asarray(np.array(cderi_sparse[p0:p1], copy=True, order='C'))
Result After Workaround
After applying both changes, the SCF calculation gives the correct result.
E1 = -1886.2263122955874 Ecoul = 534.0687392940991 Exc = -364.5454736907426
init E= -2671.33152512255
cond(S) = 773470.6758133224
The cif file is
# generated using pymatgen
data_SiC62
_symmetry_space_group_name_H-M 'P 1'
_cell_length_a 7.13600000
_cell_length_b 7.13600000
_cell_length_c 7.13600000
_cell_angle_alpha 90.00000000
_cell_angle_beta 90.00000000
_cell_angle_gamma 90.00000000
_symmetry_Int_Tables_number 1
_chemical_formula_structural SiC62
_chemical_formula_sum 'Si1 C62'
_cell_volume 363.38293146
_cell_formula_units_Z 1
loop_
_symmetry_equiv_pos_site_id
_symmetry_equiv_pos_as_xyz
1 'x, y, z'
loop_
_atom_site_type_symbol
_atom_site_label
_atom_site_symmetry_multiplicity
_atom_site_fract_x
_atom_site_fract_y
_atom_site_fract_z
_atom_site_occupancy
C C0 1 0.12294326 0.37534817 0.36906195 1
C C1 1 0.13041920 0.37928118 0.86958079 1
C C2 1 0.12341567 0.87764427 0.37266968 1
C C3 1 0.12690288 0.87883343 0.87309711 1
C C4 1 0.62805202 0.36111897 0.37194794 1
C C5 1 0.63093801 0.37534817 0.87705673 1
C C6 1 0.62386126 0.87278066 0.37613870 1
C C7 1 0.62733029 0.87764427 0.87658433 1
C C8 1 0.00157569 0.00157569 -0.00157569 1
C C9 1 0.00081094 0.00081094 0.49818735 1
C C10 1 0.00081094 0.50181265 -0.00081094 1
C C11 1 -0.00012198 0.50083957 0.49916043 1
C C12 1 0.50181265 0.00081094 -0.00081094 1
C C13 1 0.50083957 -0.00012198 0.49916043 1
C C14 1 0.50083957 0.50083957 0.00012198 1
C C15 1 0.12690288 0.12690288 0.12116656 1
C C16 1 0.13041920 0.13041920 0.62071879 1
C C17 1 0.12341567 0.62733029 0.12235572 1
C C18 1 0.12294326 0.63093801 0.62465180 1
C C19 1 0.62733029 0.12341567 0.12235572 1
C C20 1 0.63093801 0.12294326 0.62465180 1
C C21 1 0.62386126 0.62386126 0.12721934 1
C C22 1 0.62805202 0.62805202 0.63888100 1
C C23 1 -0.00068804 0.24966526 0.24641310 1
C C24 1 0.00413109 0.25310400 0.74689599 1
C C25 1 -0.00192854 0.75006920 0.24993078 1
C C26 1 -0.00068804 0.75358689 0.75033472 1
C C27 1 0.50160940 0.24606539 0.24596598 1
C C28 1 0.51585589 0.24889183 0.75110816 1
C C29 1 0.49767349 0.74654594 0.25345404 1
C C30 1 0.50160940 0.75403400 0.75393459 1
C C31 1 0.37621678 0.37621678 0.12287473 1
C C32 1 0.37524496 0.87625020 0.12374979 1
C C33 1 0.37621678 0.87712526 0.62378319 1
C C34 1 0.87625020 0.37524496 0.12374979 1
C C35 1 0.87712526 0.37621678 0.62378319 1
C C36 1 0.87544109 0.87544109 0.12455890 1
C C37 1 0.87625020 0.87625020 0.62475501 1
C C38 1 0.24966526 -0.00068804 0.24641310 1
C C39 1 0.25310400 0.00413109 0.74689599 1
C C40 1 0.24606539 0.50160940 0.24596598 1
C C41 1 0.24889183 0.51585589 0.75110816 1
C C42 1 0.75006920 -0.00192854 0.24993078 1
C C43 1 0.75358689 -0.00068804 0.75033472 1
C C44 1 0.74654594 0.49767349 0.25345404 1
C C45 1 0.75403400 0.50160940 0.75393459 1
C C46 1 0.37534817 0.12294326 0.36906195 1
C C47 1 0.37928118 0.13041920 0.86958079 1
C C48 1 0.36111897 0.62805202 0.37194794 1
C C49 1 0.37534817 0.63093801 0.87705673 1
C C50 1 0.87764427 0.12341567 0.37266968 1
C C51 1 0.87883343 0.12690288 0.87309711 1
C C52 1 0.87278066 0.62386126 0.37613870 1
C C53 1 0.87764427 0.62733029 0.87658433 1
C C54 1 0.25310400 0.25310400 -0.00413109 1
C C55 1 0.24889183 0.24889183 0.48414411 1
C C56 1 0.24966526 0.75358689 0.00068804 1
C C57 1 0.24606539 0.75403400 0.49839060 1
C C58 1 0.75358689 0.24966526 0.00068804 1
C C59 1 0.75403400 0.24606539 0.49839060 1
C C60 1 0.75006920 0.75006920 0.00192854 1
C C61 1 0.74654594 0.74654594 0.50232651 1
Si Si62 1 0.43846294 0.43846294 0.56153706 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.