Does xtuner support DPO for InternVL?
Open
- Dominant language
- Python
- Stars
- 5.2k
- Forks
- 448
- Avg merge
- 3d 15h
- Merged PRs (30d)
- 26
Description
I am trying to do a custom DPO fine-tuning for `internvl_v2_internlm2_2b_lora_finetune`, but the default config is oriented towards vanilla supervised fine-tuning with images. I tried to compare / incorporate changes from `internlm2_chat_1_8b_dpo_full` but am running into some issues with the dataset formats supported.
Is this something that `xtuner` actually supports at the moment?
Contributor guide
Assessment
This issue has not been assessed yet.