modelscope / modelscope/DiffSynth-Studio
关于Z-Image Omni代码中的文本编码长度问题
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 13h 12m
- Merged PRs (30d)
- 45
Description
def _pad_with_ids(
self,
feat: torch.Tensor,
pos_grid_size: Tuple,
pos_start: Tuple,
device: torch.device,
noise_mask_val: Optional[int] = None,
):
"""Pad feature to SEQ_MULTI_OF, create position IDs and pad mask."""
ori_len = len(feat)
pad_len = (-ori_len) % SEQ_MULTI_OF
total_len = ori_len + pad_len
# Process captions
for j, cap_item in enumerate(all_cap_feats[i]):
noise_val = images_noise_mask[i][j] if j < len(images_noise_mask[i]) else 1
cap_out, cap_pos, cap_mask, cap_len, cap_nm = self._pad_with_ids(
cap_item,
(len(cap_item) + (-len(cap_item)) % SEQ_MULTI_OF, 1, 1),
(cap_cu_len, 0, 0),
device,
noise_val,
)
cap_feats_list.append(cap_out)
cap_pos_list.append(cap_pos)
我在研究Omni源码时候发现,对于文本编码的特征。_pad_with_ids函数入参的时候就已经对齐了SEQ_MULTI_OF=32的倍数,但是进入之后还是会pad_len = (-ori_len) % SEQ_MULTI_OF继续对齐,导致输出cap_out和cap_pos的长度并不一样,这是否有问题。同时我看图像的vae和siglip特征并没有这个问题
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at _pad_with_ids and the caption-processing loop shown in the issue. Trace the lengths passed for pre-aligned text features and compare cap_out with cap_pos, then inspect the corresponding VAE and SigLIP paths. Done means confirming whether the mismatch is real and documenting or correcting it with a focused regression check.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100