XTC in DeepSpeed Compression does not work

Hi all,

Thanks for great works.

I ran some experiments with Deepspeed compression using configs in model_compression/bert. I got some issues:

• Size of output model when using DeepSpeedExamples/model_compression/bert/bash_script/XTC/quant_1bit.sh config is the same with orginal Bert model. While in the blog (https://www.deepspeed.ai/tutorials/model-compression/#3-tutorial-for-xtc-simple-yet-effective-compression-pipeline-for-extreme-compression) shows the 1bit/2bit model size reduce 32 times comparing to original model. The same with model using DeepSpeedExamples/model_compression/bert/bash_script/XTC/layer_reduction_1bit.sh
• The finetune speed of 1bit/2bit models are slower than original model.

Could you guys help to point out some reason for my result? Thanks.

Start by reproducing the reported results with model_compression/bert/bash_script/XTC/quant_1bit.sh and layer_reduction_1bit.sh, comparing output model sizes and finetuning speed with the original BERT model. Check the configurations against the XTC tutorial's expected 1-bit and 2-bit compression results; done means identifying and correcting the cause or documenting why the observed results differ.