allenai / allenai/bilm-tf

[Question] One Sentence Perplexity Computation Recommendation/Approach

Ouverte
#137 7 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
1.6k
Forks
447
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

So this might be already answered, and I mean this might be a novice question.

But I simply wanted to compute the **perplexity of a single sentence**, what parameters would I change to compute the score.

My current approach:
I still use .bin/run_test.py
However, I redirect the --test_prefix to point to a new folder with only 1 file, with one sentence in it.
I have to add the batch_size flag to roughly 1 (i.e. --batch_size 1), which computes a batch_perplexities and respectively they're avg_perplexity.

I can't help but can an itch and wonder, about the unroll_step and if I'm truely computing the perplexity of the sentence in the file.

Let me know if what I've done is accurate, or if I need to change something to properly reflect 1 sentence perplexity computation.

I've attached a run with my current model and output of my current options configurations :)

screen shot 2018-12-03 at 9 30 43 am

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.