ByteDance-Seed / ByteDance-Seed/Bagel
Support for Text-Image Interleaved Output?
- Dominant language
- Python
- Stars
- 6.2k
- Forks
- 545
- PR merge metrics
- No merged PRs in 30d
Description
Thanks for your brilliant work on Bagel—it's an impressive and inspiring project!
I’m wondering whether Bagel currently supports (or plans to support) text-image interleaved output, as described in Section 3.3 "Vision-Text Interleaved Data" of the paper.
For example, can the model perform inference like the following:
Input:
```
Use pictures and words to explain each step to cook eggs.
```
Output:
```
[image1]
Step 1: Crack the eggs into a bowl and whisk them together with a fork or a whisk.
[image2]
Step 2: Heat a non-stick pan or skillet over medium heat.
[image3]
Step 3: Once the oil is hot, pour the eggs into the pan and use a spatula to gently scramble them.
[image4]
Step 4: Once the eggs are cooked to your desired doneness, season with salt and pepper to taste.
[image5]
Step 5: Once the eggs are cooked to your liking, transfer them to a serving plate and enjoy!
```
If this isn't currently supported, I’d be curious to know whether it's feasible to extend Bagel to this use case, and if you have any recommendations on how to approach it with the existing codebase.
Thanks again for your amazing work—really excited to see how this project evolves!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading Section 3.3, “Vision-Text Interleaved Data,” of the referenced paper and compare its requirements with Bagel’s current multimodal inference behavior. The issue names no files or tests; completion would require a scoped design and a demonstrated decision about whether interleaved text-image output is supported or feasible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100