facebookresearch / facebookresearch/SpinQuant

Questions about replication experiments.

Open
#5 5 comments 9 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
428
Forks
93
PR merge metrics
No merged PRs in 30d

Description

Are the code and parameters in this repository consistent with the parameters used in the experiments described in the paper? I conducted an experiment on an A100 using the provided command "bash 10_optimize_rotation.sh meta-llama/Llama-2-7b 4 4 4" keeping the parameters unchanged in the script. However, the results I obtained differ somewhat from the results reported in the paper.
This is my command "bash 10_optimize_rotation.sh meta-llama/Llama-2-7b 4 4 4", and I only change the nproc_per_node to 1. When I use the eval command "2_eval_ptq.sh $model_name 4 4 4", the ppl is about 8.49.
The model I use is NousResearch/Llama-2-7b-chat-hf.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.