lllyasviel / lllyasviel/sd-forge-layerdiffuse

Problems with Iterative Generation

Open
#83 0 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.1k
Forks
352
PR merge metrics
No merged PRs in 30d

Description

I've been extensively experimenting with this incredible project and have found most implementations to function seamlessly. However, I've encountered several issues specifically related to iteration generation that I hope to get some guidance on or report for potential fixes.

**Object Positioning in Iterations**: It appears that the model struggles to modify the position of objects in accordance with the prompts provided. The objects are always generated in the middle of the image during my observation.

**Background Fidelity Across Iterations**: I've observed that the background tends to become faded or lose detail during the iterative process, a phenomenon that can be detected even within a single iteration cycle.

**Unexpected Object Generation**: When using prompts that specify object relationships, such as 'box in front of man' or 'a book in front of a cat', the model occasionally introduces extra, unintended objects into the scene, which you can see in the pictures below.

Given these observations, I'm wondering if there might be adjustments I could make on my end to mitigate these issues, or if they are known limitations or bugs within the model itself.

![image](https://github.com/layerdiffusion/sd-forge-layerdiffuse/assets/145271686/2db07c85-cebd-4c5e-bbf9-782649a6eac2)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported iterative-generation behavior using the example prompts, including object positioning, background fidelity, and unintended objects. No source file or test is identified; done would require determining whether these observations are reproducible bugs or known model limitations and documenting the result.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.