modelscope / modelscope/DiffSynth-Studio

关于 Flux-Kontext 支持多张 kontext_images 输入的设计疑问

Open
#686 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.1k
Forks
1.3k
Avg merge
13h 12m
Merged PRs (30d)
45

Description

您好,感谢你们开源 DiffSynth-Studio 项目!

我在阅读 flux_image_new.py 中 FluxImageUnit_Kontext 的实现时注意到,当前代码支持输入多张 kontext_images,并将多个 kontext_latent 在非通道维度(dim=1)进行拼接:

kontext_latents = torch.concat(kontext_latents, dim=1)
kontext_image_ids = torch.concat(kontext_image_ids, dim=-2)

这似乎与 Flux-Kontext 官方开源实现(只支持单张图)不完全一致。在我们的测试中,尝试输入多张图像时,输出结果只是简单地将图像融合,整体表现并不协调。

因此我有几个疑问想请教:

  1. 当前支持多张 kontext_images 的设计初衷是什么?是为后续支持多图编辑、多图融合等扩展功能做准备吗?
  2. 将多个 latent 在非通道维度拼接是否符合模型结构本身的预期?这种拼接方式是否可能会引起空间布局的错乱或 feature 混淆?
  3. 若我们希望通过 SFT(Supervised Fine-Tuning)方式提升多图输入的效果,是否有推荐的数据量?你们是否有这方面的经验可以分享?

再次感谢你们的工作,期待解答!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with diffsynth/pipelines/flux_image_new.py and the FluxImageUnit_Kontext implementation, then compare its multi-image latent concatenation with the official Flux-Kontext behavior described in the issue. There is no concrete change or acceptance criterion yet; a maintainer decision is needed before the work can be scoped or considered done.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.