ByteDance-Seed / ByteDance-Seed/Bagel
Parallel cfg forward
- Dominant language
- Python
- Stars
- 6.2k
- Forks
- 545
- PR merge metrics
- No merged PRs in 30d
Description
Hi guys! Thanks so much for your work!
I wonder if it is possible to make cfg forward in a parallel way, so that input sequences with dropped conditions and the initial sequence are fed in a batched manner to the llm backbone?
Is it possible to give batched sequences as an input to Qwen2Model?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the CFG forward path and the Qwen2Model entry point to determine how dropped-condition and initial sequences are currently passed to the LLM backbone. Check whether Qwen2Model already accepts batched sequences, then define completion as parallel batched CFG execution with behavior matching the existing sequential path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100