lllyasviel / lllyasviel/FramePack

Suggestion: Improve the generation preview

Open
#239 1 comment 3 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
17.3k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

I'm talking about the "Next Latents" block.

1. Fix the wrongly clamped colors:
In hunyuan.py in the function `vae_decode_fake`:
```diff
- images = images.clamp(0.0, 1.0)
+ images = images.clamp(-1.0, 1.0)
```
In demo_gradio.py:
```diff
- preview = (preview * 255.0).detach().cpu().numpy().clip(0, 255).astype(np.uint8)
+ preview = (preview * 127.5 + 127.5).detach().cpu().numpy().clip(0, 255).astype(np.uint8)
```

2. Output the preview as an animation instead of a frame strip, as gleaming what motion is about to be generated from a frame strip can be difficult. Use APNG for this, for example, it's perfect for the job considering the small size of the preview, and MP4 would probably wreck it with compression artifacts.
Won't share the code for this, as my knowledge of python is near non-existent to write it myself, and the code ChatGPT generated is, while functional, completely deranged and full of temporary files, wasting hard drive space and probably not even closing handles.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with vae_decode_fake in hunyuan.py and the preview conversion in demo_gradio.py, checking how the “Next Latents” block currently produces its frame strip. Correct the preview color range and replace the strip with a compact APNG animation. Done means the preview colors are no longer wrongly clamped and the block displays motion as an animation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, computer-graphics
Issue type
Feature
Difficulty
3/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.