NVIDIA-NeMo / NVIDIA-NeMo/RL

[multimodal dtensor] Inconsistent logprobs for multimodal models

Open
#793 2 comments 0 reactions 1 assignee Claimed by @rohitrango View on GitHub
bug t-pytdensor
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

**Describe the bug**

When running GRPO for VLMs (Qwen2.5VL, LLaVa, etc.) the logprobs generated by vllm and that by huggingface differ by a margin higher than 1.05. Although the policy converges across different VLMs.

**Steps/Code to reproduce bug**

Run `uv run examples/run_vlm_grpo.py` from PR #712

**Expected behavior**

A clear and concise description of what you expected to happen.

**Environment overview (please complete the following information)**

- Environment location: [local / cluster]
- Method of install: [pip install or from source].

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.