NVIDIA / NVIDIA/Model-Optimizer
Puzzletron Progress 6/8 (calculating one block scores) takes 10 to 20 times more than in tutorial
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.8k
- Forks
- 604
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 142
Description
ModelOpt: release/0.44.0
Running torchrun --nproc_per_node 2 examples/puzzletron/main.py --config examples/puzzletron/configs/llama-3_1-8B_pruneffn_memory/llama-3_1-8B_pruneffn_memory.yaml 2>&1 | tee ./log.txt | grep "Puzzletron Progress"
takes 10 to 20 times longer than stated in the tutorial: https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/puzzletron
it is due to scoring.eval_samples: 128 in the examples/puzzletron/configs/llama-3_1-8B_pruneffn_memory/Llama-3_1-8B.yaml
suggestions:
- adjust tutorial and config file
- provide a better progress bar to indicate the remaining time
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Puzzletron tutorial and examples/puzzletron/configs/llama-3_1-8B_pruneffn_memory/Llama-3_1-8B.yaml, focusing on scoring.eval_samples, then run the provided torchrun command to compare the documented and actual timing. Done means the tutorial and configuration agree on expected runtime, with any progress indication requested by the issue addressed or clearly scoped.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- documentation, machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100