[QUESTION]NVFP4 Post SFT Training & Model Accuracy
Open
community-request
question
waiting-on-customer
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 4.5k
- Avg merge
- 4d 6h
- Merged PRs (30d)
- 271
Description
**Your question**
Hi @Phlip79 , this is regarding the issue I created earlier (see below) related to nvfp4. One of the follow on activities I am trying to do is to gauge the accuracy of the SFT trained model using nvfp4. After completion of the SFT training run using nvfp4, I tried accessing the model using SgLang inference serving engine. The output of the inference request seems to be all gibberish. It all seems to work fine without any quantization. Is there anything I am missing that is messing up the model while doing post SFT training using nvfp4? Thanks.
https://github.com/NVIDIA/Megatron-LM/issues/3470#issue-3955671018
Contributor guide
Assessment
This issue has not been assessed yet.