NVIDIA / NVIDIA/Megatron-LM

[QUESTION]NVFP4 Post SFT Training & Model Accuracy

Open
#3,671 8 comments 0 reactions 1 assignee Claimed by @guihong-nv View on GitHub
community-request question waiting-on-customer
Dominant language
Python
Stars
17.9k
Forks
4.5k
Avg merge
4d 6h
Merged PRs (30d)
271

Description

**Your question**
Hi @Phlip79 , this is regarding the issue I created earlier (see below) related to nvfp4. One of the follow on activities I am trying to do is to gauge the accuracy of the SFT trained model using nvfp4. After completion of the SFT training run using nvfp4, I tried accessing the model using SgLang inference serving engine. The output of the inference request seems to be all gibberish. It all seems to work fine without any quantization. Is there anything I am missing that is messing up the model while doing post SFT training using nvfp4? Thanks.

https://github.com/NVIDIA/Megatron-LM/issues/3470#issue-3955671018

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.