lllyasviel / lllyasviel/FramePack
Magic number 1、2、16
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 17.3k
- Forks
- 1.7k
- PR merge metrics
- No merged PRs in 30d
Description
How does the magic number 1, 2, 16 come about? Why not 1, 2, 8, or something else?
I think they are used to represent three types of history latent frame count.
The first type represents the most recent history, using 1 latent frame, and is processed using a conv kernel (1, 2, 2).
The second type represents the second most recent history, using 2 latent frames, and is processed using a conv kernel (2, 4, 4).
The third type represents the third most recent history, using 16 latent frames, and is processed using a conv kernel (4, 8, 8).
For a 5-second video, fps 30, it will be processed to 37 latent frames, which are indexed from 0 to 36. Then they will be cut to 0, 1-9, 10-18, 19-27, 28-36, namely the first frame and 4 latent sections.
If using the magic numbers 1, 2, 16, the first latent section, which is indexed 1-9, will use the history latent frame 10, 11-12, 13-28. The remaining latent frames indexed 29-36 are not used.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files or tests; start by locating the implementation of the 1, 2, and 16 history-frame values and the latent-frame partitioning it describes. Check the indexing and kernel relationships against the reported 5-second example, then document the rationale or identify the discrepancy so the history sections and unused frames are explained.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100