deepspeedai / deepspeedai/DeepSpeedExamples
Inquiry about Embedding Concatenation in DeepSpeed-VisualChat
@xiaoxiawu-microsoft is already working on this.
Since Nov 14, 2023.
- Dominant language
- Python
- Stars
- 6.8k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 1
Description
@yaozhewei First, I'd like to extend my gratitude for the incredible work you've been doing with DeepSpeedExamples. It's truly commendable and has been a great resource for the community.
As I was exploring the code, particularly the section where language and visual embeddings are concatenated, I came across something that prompted a question. I noticed in this line of code:
that img_pos_list is reversed during the insertion of the image embedding. However, it appears that cur_img is not reversed in the process. Could there potentially be a mismatch between the visual and text information due to this? I'm curious if this is intentional for alignment purposes or if it might be an oversight.
I would appreciate any clarification you can provide on this matter.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.