lllyasviel / lllyasviel/FramePack

Questions about changing the history length, and the training cose of framepack

Open
#791 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
17.3k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

Dear authors,

Thanks for your inspiring work! I read through your code and paper and found that the history length used in the code implementation is 1+2+16=19 latent frames. The paper said the method should support longer length, so I am wondering if variable length of history frames can be supported by finetuning the currently provided framepack model with e.g. LoRA training?

Also, I have a question about the training cost. In the paper, you said "FramePack achieves a batch size of 64 on a single 8×A100-80G node with the
13B HunyuanVideo model at 480p resolution". Do you mean full-parameter training of framepack with 13B Hunyuan base model can be trained with 8*80G GPUs? I have tested LoRA training using code from [musubi-tuner](https://github.com/[kohya-ss/musubi-tuner](https://github.com/kohya-ss/musubi-tuner/tree/main)/tree/main), but it cost about 38G for training lora only.

I hope you can help about these questions. Thank you so much!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the history-length implementation described in the FramePack code with the paper's claim about longer histories. Then review the linked musubi-tuner LoRA training code and the paper's 13B HunyuanVideo, 480p, and 8×A100-80G training details; done means documenting whether variable history can be fine-tuned and clarifying the stated training setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.