InternLM / InternLM/InternLM-XComposer

interleav_wrap has no padding bug

Open
#202 4 comments 1 reaction 1 assignee Assigned to @LightDXY View on GitHub
Dominant language
Python
Stars
2.9k
Forks
175
PR merge metrics
No merged PRs in 30d

Description

![image](https://github.com/InternLM/InternLM-XComposer/assets/18352727/6aa60541-3aa9-4d42-83af-67db53c9e2ab)
There is no padding when wrap_embeds = torch.cat(wrap_embeds_list) in interleav_wrap function, Is this a bug in it?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.