[Question] One Sentence Perplexity Computation Recommendation/Approach
- Lingua principale
- Python
- Stelle
- 1.6k
- Fork
- 447
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
So this might be already answered, and I mean this might be a novice question.
But I simply wanted to compute the **perplexity of a single sentence**, what parameters would I change to compute the score.
My current approach:
I still use .bin/run_test.py
However, I redirect the --test_prefix to point to a new folder with only 1 file, with one sentence in it.
I have to add the batch_size flag to roughly 1 (i.e. --batch_size 1), which computes a batch_perplexities and respectively they're avg_perplexity.
I can't help but can an itch and wonder, about the unroll_step and if I'm truely computing the perplexity of the sentence in the file.
Let me know if what I've done is accurate, or if I need to change something to properly reflect 1 sentence perplexity computation.
I've attached a run with my current model and output of my current options configurations :)

Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Valutazione
Questa issue non è ancora stata valutata.