mlpstorage incorrectly identifies llama3-8b on 16 GPU as qualified CLOSED submission
Open
Nobody has claimed this yet.
DLIO or mlpstorage
- Dominant language
- Python
- Stars
- 203
- Forks
- 67
- Avg merge
- 20m
- Merged PRs (30d)
- 8
Description
See the log below. It should be set as OPEN
2025-06-20 09:55:15|STATUS: Benchmark results directory: ./results/eagle/n2x8/checkpointing/llama3-8b/20250620_095514
2025-06-20 09:55:15|INFO: Found benchmark run: checkpointing_run_llama3-8b_20250620_095514
2025-06-20 09:55:15|STATUS: Verifying benchmark run for checkpointing_run_llama3-8b_20250620_095514
2025-06-20 09:55:15|STATUS: Benchmark run qualifies for CLOSED category ([RunID(program='checkpointing', command='run', model='llama3-8b', run_datetime='20250620_095514')])
2025-06-20 09:55:15|WARNING: Running the benchmark without verification for open or closed configurations. These results are not valid for submission. Use --open or --closed to specify a configuration.
2025-06-20 09:55:15|STATUS: Instantiated the Checkpointing Benchmark...
2025-06-20 09:55:15|STATUS: Running benchmark command:: mpiexec -n 16 --ppn 8 --cpu-bind depth -d 16 /home/hzheng/crux/DLIO/dlio_benchmark/workspace/2025-06-19/pyenvs/2025-06-19/storage/bin/dlio_benchmark workload=llama3_8b ++hydra.run.dir=./results/eagle/n2x8/checkpointing/llama3-8b/20250620_095514 ++hydra.output_subdir=dlio_config ++workload.checkpoint.num_checkpoints_read=0 ++workload.checkpoint.num_checkpoints_write=10 ++workload.checkpoint.checkpoint_folder=.//checkpoints/n2x8/llama3-8b --config-dir=/lus/eagle/projects/PolarisAT/hzheng/crux/DLIO/dlio_benchmark/workspace/2025-06-19/storage/configs/dlio
@wvaske Could you please address this?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the verification and qualification path that emits the logged “qualifies for CLOSED category” message, using the llama3-8b 16-GPU run shown here as the reproduction case. Trace why it selects CLOSED and verify that this case is classified as OPEN; the issue provides no file or test name to narrow the search.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100