allenai / allenai/bilm-tf

[Question] One Sentence Perplexity Computation Recommendation/Approach

Aperta
#137 7 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
1.6k
Fork
447
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

So this might be already answered, and I mean this might be a novice question.

But I simply wanted to compute the **perplexity of a single sentence**, what parameters would I change to compute the score.

My current approach:
I still use .bin/run_test.py
However, I redirect the --test_prefix to point to a new folder with only 1 file, with one sentence in it.
I have to add the batch_size flag to roughly 1 (i.e. --batch_size 1), which computes a batch_perplexities and respectively they're avg_perplexity.

I can't help but can an itch and wonder, about the unroll_step and if I'm truely computing the perplexity of the sentence in the file.

Let me know if what I've done is accurate, or if I need to change something to properly reflect 1 sentence perplexity computation.

I've attached a run with my current model and output of my current options configurations :)

screen shot 2018-12-03 at 9 30 43 am

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.