facebookresearch / facebookresearch/blt

A question about training process of Entropy Model.

Open
#100 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.1k
Forks
196
PR merge metrics
No merged PRs in 30d

Description

Hellow! I am curious about your work. I have some question when I am reading the paper.

1. How do you train the entropy model? Do you train it on the whole dataset the same as the latent model, or you just divide a small dataset for training entropy model and then use it to predict entropies of the remain data? If the entropy model is trained directly on the entire dataset, can the entropy model still estimate the entropy of the training set(I mean the training set of latent model) well? The entropy model should give a smaller entropy estimate due to the exposure of the data, right?
2. How do you design the complexity of the entropy model? Do you design the entropy model strictly according to, for example, a few percent of the parameters of the latent model?

Sincerely waiting for your reply :)

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. Start by reviewing the paper and the repository's entropy-model and latent-model training documentation, then provide answers about dataset use, generalization to training data, and model-complexity choices.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.