modelscope / modelscope/DiffSynth-Studio
How to combine Inpaint and ContentRef templates?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 13h 12m
- Merged PRs (30d)
- 45
Description
Hi, thanks for this great project.
I am trying to use these two templates together:
DiffSynth-Studio/Template-KleinBase4B-InpaintDiffSynth-Studio/Template-KleinBase4B-ContentRef
My goal is:
- Use Inpaint to modify only the masked area.
- Use ContentRef to guide the generated content/style.
Both templates work individually, but I am not sure how to combine them correctly.
I tried loading both templates:
template = TemplatePipeline.from_pretrained(
torch_dtype=torch.bfloat16,
device="cuda",
model_configs=[
ModelConfig(model_id="DiffSynth-Studio/Template-KleinBase4B-Inpaint"),
ModelConfig(model_id="DiffSynth-Studio/Template-KleinBase4B-ContentRef"),
],
)
and passing both inputs:
template_inputs=[
{
"model_id": 0,
"image": image,
"mask": mask,
"force_inpaint": True,
},
{
"model_id": 1,
"image": reference_image,
},
]
But I got:
paste error or describe the unexpected result here
- Is combining Inpaint + ContentRef supported?
- Is this the correct way to pass multiple templates?
- Are there any additional settings required for ContentRef + Inpaint?
Thanks!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the TemplatePipeline.from_pretrained entry point and the ModelConfig and template_inputs usage shown in the issue. Check whether Inpaint and ContentRef are supported together and what the completed input should be; done means a reproducible supported example or a clear limitation and error explanation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100