kohya-ss / kohya-ss/sd-scripts
SDXL Training prompt length limited to 225 tokens or can be more?
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
Hi, is the training prompt length limited to 225 tokens or can be more?
[here](https://github.com/kohya-ss/sd-scripts/blob/main/docs/fine_tune_README_ja.md#%E3%83%88%E3%83%BC%E3%82%AF%E3%83%B3%E9%95%B7%E3%81%AE75%E3%81%8B%E3%82%89225%E3%81%B8%E3%81%AE%E6%8B%A1%E5%BC%B5) from the automated translation, i understand that the CLIP limit is 75 tokens, but is extended to 225 by splitting in chunks.
In other places there seems to be no limit to chunks, and prompt length, and it is not clear if there is a reason to limit it to 3 chunks.
So how is currently the limit in the code?
Can it be extended?
Should i care at which point the prompt is split? Is better for for related tokens to be in the same chunk (sentence not split) or it doesn't matter?
Thanks!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with docs/fine_tune_README_ja.md at the section describing expansion from 75 to 225 tokens, then trace the repository code that handles token chunks for SDXL training. Determine the current limit, whether longer prompts are supported, and how chunk boundaries affect related tokens before defining any change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100