Partial clothes like a skirt gets transferred from driving video to reference image in final output.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 90
- Avg merge
- 19h 19m
- Merged PRs (30d)
- 1
Description
Fantastic work! I've been playing with SCAIL2 for the past two weeks and it is the best available!!
There is one issue I'm trying to solve or understand. I have a driving video where a dancer wearing a yellow tutu/skirt (flange material hanging down like a ballet dancer clothes). The tutu part of the skirt (lower part of the round material) is getting transferred to my reference photo dancer lower body/butt area which is naked. It suddenly appears mid video or sometimes randomly and soon after few movements it disappears/fades away. I only want to keep the background from the driving video and my own character just like in the reference image and the dance motion (pose).
My reference image already has a black background(removed previously with specific BG Remove node), Replace Mode options are OFF on both SAM3 colored mask node and WanSCAILtoVideo node. Everything is great except part of that yellow skirt keeps transferring to my character in the final video (fades in and then out). I tried to specify/exclude it in prompts, positive and negative, tried to lower/increase lora strength (lightx2v I2V 480p rank256/128/64 tried T2V different lora also)
I also used the video prompt in SAM3 to include skirt or not include where I only mention detect just "person" and still not able to get rid of this effect. I tried various Clip Vision models and the effect still happens.
Other less important issue:
In addition to this effect sometimes I get teleporting, where the subject form reference image gets teleported back to reference image initial position, or just randomly closer to the viewer. Fades out and fades in closer randomly. (especially on long videos)
I was able to fix this by resizing driving video or sometimes resizing the reference image playing with video/image sizes/crop.
The main issue is the clothes transfer from driving video to final video. How can I fix this? Any help would be appreciated.
Let me know which parts of the workflow/models are responsible for this.
Thank you!!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the SAM3 colored mask and WanSCAILtoVideo nodes, then compare the effect across the Clip Vision models and LoRA settings described in the report. Determine which workflow or model component introduces the skirt and teleporting artifacts, and confirm that the background and pose remain intact without transferring clothing from the driving video.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100