No such file or directory: "./hf_model_safetensors/model.safetensors"

root@fe6be27dc3d5:/share/repo/picotron# torchrun --nproc_per_node 4 train.py --config tmp/llama-1B-cpu/config.json
[2025-04-21 17:31:15,941] torch.distributed.run: [WARNING]
[2025-04-21 17:31:15,941] torch.distributed.run: [WARNING] *****************************************
[2025-04-21 17:31:15,941] torch.distributed.run: [WARNING] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
[2025-04-21 17:31:15,941] torch.distributed.run: [WARNING] *****************************************
rank 0: Creating tokenizer
rank 0: Broadcasting tokenizer to all ranks
Model /share/_model_/SmolLM-1.7B already exists at /share/_model_/SmolLM-1.7B
init dataloader time: 0.38s
rank 0: Creating model config
rank 0: Broadcasting model_config to all ranks
rank 2: Initializing model meta device
Traceback (most recent call last):
File "/share/repo/picotron/train.py", line 183, in
model = init_model_with_materialized_weights(model, model_config, save_dir=f"./hf_model_safetensors/")
File "/share/repo/picotron/picotron/checkpoint.py", line 79, in init_model_with_materialized_weights
with safe_open(safetensors_path, framework="pytorch", device="cpu") as f:
FileNotFoundError: No such file or directory: "./hf_model_safetensors/model.safetensors"

I mkdir hf_model_safetensors, but error still exist

Start with train.py around line 183 and follow init_model_with_materialized_weights in picotron/checkpoint.py to understand how ./hf_model_safetensors/model.safetensors is expected to be produced or located. Re-run the reported torchrun command and inspect the model and save-directory setup. Done means the command can find the expected safetensors file or the issue clearly identifies the missing prerequisite.

Written by the indexing model from the issue text.