ai-forever / ai-forever/ru-gpts

Correct data format for fine-tuning RUGPT3 models

Aperta
#109 0 commenti 1 reazione 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
2.1k
Fork
433
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

Hello!

I'm learning how to fine-tune RuGPT3 models with my own dataset to generate similar texts. I'm wondering if there is the documentation describing the right dataset format and a list of special tokens.

The specific questions are following:
1. The problem is that in my dataset there are both one-line and multiline samples, and I'm wondering how to separate them from each other, as it seems to be assumed that new line is a separator by default.
2. All the texts in my corpus are of the same type (for example, let's say that these are jokes, but they cannot be combined on some big topics) and I want to generate a new text without a specific input, e.g. I don't assume to give beginning of some text. Should I use in my dataset a keyword like "Анекдот", e.g. "Анекдот: ", and then use this keyword as a prompt? If so, do I need some special token for that word?

I would be grateful for any information on the data format.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.