多模态多轮对话RLHF数据集如何构建

{
  "messages": [
    {"role": "user", "content": "<image>...<image>请根据提供的甲状腺超声图像..."},
    {"role": "assistant", "content": "\n穿刺会诊：\n\n左叶下部内侧见低回声..."},
    {"role": "user", "content": "基于这些图像和上述超声所见，得出检查诊断是什么？"},
    {"role": "assistant", "content": "\n甲状腺多发实性结节，右叶者可行穿刺\n\n"},
    {"role": "user", "content": "综合这些图像...穿刺会诊结果为阳性还是阴性？"},
    {"role": "assistant", "content": "阳性"}
  ],
  "images": ["./data_filtered/train/.../1.jpg", "..."],
  
  "rejected_messages": [
    {"role": "user", "content": "<image>...<image>请根据提供的甲状腺超声图像..."},
    {"role": "assistant", "content": "\n穿刺会诊：\n\n左叶见等回声..."},
    {"role": "user", "content": "基于这些图像和上述超声所见，得出检查诊断是什么？"},
    {"role": "assistant", "content": "\n甲状腺单发囊性结节，建议随访\n\n"},
    {"role": "user", "content": "综合这些图像...穿刺会诊结果为阳性还是阴性？"},
    {"role": "assistant", "content": "阴性"}
  ],
  "rejected_images": ["./data_filtered/train/.../1.jpg", "..."]
}

上诉构建的RLHF数据格式是否正确？

The issue gives a proposed multimodal, multi-turn RLHF record with accepted and rejected messages and image paths, but names no files, tests, or entry points. Start by locating the repository's RLHF dataset-format documentation and loader entry point, then compare this schema with the format actually accepted for multimodal preference data. Done means confirming the required fields and any needed changes to the example.