deepmodeling / deepmodeling/deepmd-kit
[BUG]_SSHFS_incompatibility_issue_with_DeepMD_3.1.2
- Dominant language
- Python
- Stars
- 2k
- Forks
- 649
- Avg merge
- 6d 18h
- Merged PRs (30d)
- 15
Description
### Bug summary
Hello There,
i am currently training a model using Deepmd 3.1.2 with dpgen and I am experiencing issue when training on a SSHFS mounted system. The reported error seems to have a problem regarding the PyTorch workers accessing the same file at the same time.
I have attached the complete message regarding the problem. I am wondering if this has something to do with the Deepmd 3.1.2 or the SSHFS 3.7.3?
### DeePMD-kit Version
3.1.2
### Backend and its version
PyTorch
### How did you download the software?
conda
### Input Files, Running Commands, Error Log, etc.
[Pasted text(20260908-131850).txt](https://github.com/user-attachments/files/32094818/Pasted.text.20260908-131850.txt)
### Steps to Reproduce
train normally on a SSHFS mounted system with the SSHFS version 3.7.3.
### Further Information, Files, and Links
_No response_
Contributor guide
Research direction
Start with Pasted.text.20260908-131850.txt and the reported training setup: DeePMD-kit 3.1.2, PyTorch, dpgen, and SSHFS 3.7.3. Reproduce training on an SSHFS-mounted system and determine whether concurrent worker access causes the reported error; done means identifying a reproducible incompatibility or a confirmed external cause.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100