ByteDance-Seed / ByteDance-Seed/Bagel
Drop rates
- Dominant language
- Python
- Stars
- 6.2k
- Forks
- 545
- PR merge metrics
- No merged PRs in 30d
Description
Hi, guys! In your paper you write that you drop text, vit and vae tokens with probs 0.1, 0.5 and 0.1. Is this true, or this is a typo, and you dropped vae tokens with 0.5 rate?
I am now trying to make a pretrain with these settings, and it looks like model is overconditioned on vae tokens, and without them (on t2i) quality degrades during train.
Contributor guide
No contributing guide indexed for this repository
Research direction
Compare the paper's stated text, ViT, and VAE token-drop probabilities (0.1, 0.5, and 0.1) with the pretraining settings used by Bagel. Confirm whether the VAE rate is 0.1 or 0.5, then document the authoritative value and its effect on the t2i setup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100