lllyasviel / lllyasviel/FramePack

Fails to follow prompts to generate complex sequential actions

Open
#28 35 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
17.3k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

Hello lvmin,

First, thank you for developing this fantastic tool! I’m really excited about the possibilities it offers for video generation.

I’ve been experimenting with prompts that describe multiple actions in sequence, but I’ve noticed that the generated video tends to ignore everything after the first action and simply loops that initial movement. I was hoping the model would perform each step in order, but it seems to fall back to repetition.

Prompt Used:

The woman dressed in green took a sip of water, then stood up and picked up the picture frame on the table and looked at it, leaving tears in her eyes.

Observed Output:

  • The video only shows the woman taking a sip of water on repeat, without ever standing up or picking up the frame.

Image

So Is there a recommended way to structure multi-step prompts so that FramePack follows each action in sequence? Or are there any known limitations or upcoming features planned to improve narrative coherence across frames?

Thank you in advance for your time and assistance. I appreciate any guidance you can provide to help me achieve more accurate, sequential action in my generated videos!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files, tests, or entry points. Start by reproducing the supplied prompt and reviewing the project’s prompt and video-generation paths to determine whether sequential actions are supported. Done would require a clearly defined fix or documented limitation, but the issue currently asks for guidance and future plans rather than a scoped change.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.