Does it support Preference data (for training Reward / DPO)?

🚀 Feature Request

The preference data looks like this:

{
    "chosen":
    [
        {"role": "user", "content": "abcd"},
        {"role": "assistant",  "content": "abcef"},
        ...
    ],
    "rejected":
    [
        {"role": "user", "content": "abcd"},
        {"role": "assistant", "content": "abcef"},
        ...
    ]
}

This data is used to train a Reward Model or DPO

I'm wondering if it's possible to use streaming for this kind of situation. And How?
Thanks very much.

The issue names no files, tests, or entry points. Start by reviewing the library's dataset streaming interface and how it represents records, then determine whether chosen/rejected conversation pairs can be streamed for Reward Model or DPO training. Done means the supported data shape and usage path are documented or implemented, with relevant coverage.