NVIDIA-NeMo / NVIDIA-NeMo/RL

Automodel: CombinedProjectionStateDictAdapter missing convert_single_tensor_to_hf

Open
#2,072 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2k
Forks
561
Avg merge
4d 5h
Merged PRs (30d)
145

Description

## Summary

Custom model implementations that use `CombinedProjectionStateDictAdapter` (e.g. Qwen2, Llama) do not implement `convert_single_tensor_to_hf`, which is required for per-tensor weight syncing in `_maybe_adapt_tensor_to_hf` (`dtensor_policy_worker_v2.py`).

This causes a runtime error when using these models with LoRA weight syncing (e.g. `grpo_automodel_lora` tests).

## Current Workaround

Users can set `policy.dtensor_cfg.automodel_kwargs.force_hf=true` to bypass the custom model implementation and use HuggingFace's native implementation, which doesn't need a `StateDictAdapter`.

We've also added a fail-fast check in `setup.py` to detect this at model init time and surface a helpful error message.

## Proper Fix

The upstream fix should be in Automodel — either:
1. Implement `convert_single_tensor_to_hf` on `CombinedProjectionStateDictAdapter` (it already has `to_hf()` which does the full conversion, so a single-tensor version should be straightforward), or
2. Have Automodel auto-fallback to HF when the adapter is incomplete.

## Affected Models

Any model whose `StateDictAdapter` inherits from `CombinedProjectionStateDictAdapter` without overriding `convert_single_tensor_to_hf`:
- `Qwen2StateDictAdapter`
- `LlamaStateDictAdapter`

## Affected Tests

- `grpo_automodel_lora`
- `grpo_automodel_lora_async`
- `grpo_automodel_lora_non_colocated`
- `grpo_rm_env`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.