bytedance / bytedance/1d-tokenizer
Why do we need two codebooks? So is this actually a 1D+2D representation learning?
- Dominant language
- Jupyter Notebook
- Stars
- 1.2k
- Forks
- 70
- PR merge metrics
- No merged PRs in 30d
Description
I initially was very excited about this paper. However, after reading the code, I found there were actually two code books and two representations, where one is 1D(K=32) and another is 2D(16x16). All the other models use one code book and one presentation, why does this use two codebooks and two representations? Why not just use the 1D codewords seq for reconstruction?
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no file, test, or entry point. Read the paper alongside the repository code implementing the 1D and 2D codebooks, then document why both representations are used and whether reconstruction can rely only on the 1D codeword sequence.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100