lllyasviel / lllyasviel/FramePack

Magic number 1、2、16

Open
#710 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
17.3k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

How does the magic number 1, 2, 16 come about? Why not 1, 2, 8, or something else?

I think they are used to represent three types of history latent frame count.
The first type represents the most recent history, using 1 latent frame, and is processed using a conv kernel (1, 2, 2).
The second type represents the second most recent history, using 2 latent frames, and is processed using a conv kernel (2, 4, 4).
The third type represents the third most recent history, using 16 latent frames, and is processed using a conv kernel (4, 8, 8).

For a 5-second video, fps 30, it will be processed to 37 latent frames, which are indexed from 0 to 36. Then they will be cut to 0, 1-9, 10-18, 19-27, 28-36, namely the first frame and 4 latent sections.
If using the magic numbers 1, 2, 16, the first latent section, which is indexed 1-9, will use the history latent frame 10, 11-12, 13-28. The remaining latent frames indexed 29-36 are not used.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files or tests; start by locating the implementation of the 1, 2, and 16 history-frame values and the latent-frame partitioning it describes. Check the indexing and kernel relationships against the reported 5-second example, then document the rationale or identify the discrepancy so the history sections and unused frames are explained.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.