About KB_size during training
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 1.5k
- Forks
- 125
- Avg merge
- 22d 8m
- Merged PRs (30d)
- 1
Description
Hello, I have a question about KB_size during training.
I found that during training, we can trace epoch-training loss graph as well as epoch-KB size graph.
I observed that during training, our loss decreases, but the KB_size is consistent.
According to the paper, the authors described "During training, we find limiting the KB size crucial for successful convergence.", indicating our model has been trained well.
However, when I saw the Issue #75, it looks like KB_size should increase.
Could you please let me know which one is right?
Also, if it should increase, could you let me know why you intentionally increases the KB_size during training?
Thanks,
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the paper's statement about limiting KB size during convergence and compare it with the epoch-training loss and epoch-KB size graphs. Then read Issue #75 and trace the training behavior that produces those graphs. Done means documenting which KB_size behavior is expected and why.
Written by the indexing model from the issue text.
Assessment
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100