facebookresearch / facebookresearch/blt
A question about training process of Entropy Model.
- Dominant language
- Python
- Stars
- 2.1k
- Forks
- 196
- PR merge metrics
- No merged PRs in 30d
Description
Hellow! I am curious about your work. I have some question when I am reading the paper.
1. How do you train the entropy model? Do you train it on the whole dataset the same as the latent model, or you just divide a small dataset for training entropy model and then use it to predict entropies of the remain data? If the entropy model is trained directly on the entire dataset, can the entropy model still estimate the entropy of the training set(I mean the training set of latent model) well? The entropy model should give a smaller entropy estimate due to the exposure of the data, right?
2. How do you design the complexity of the entropy model? Do you design the entropy model strictly according to, for example, a few percent of the parameters of the latent model?
Sincerely waiting for your reply :)
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by reviewing the paper and the repository's entropy-model and latent-model training documentation, then provide answers about dataset use, generalization to training data, and model-complexity choices.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100