lllyasviel / lllyasviel/FramePack

File "av/video/frame.pyx", line 204, in av.video.frame.VideoFrame.pict_type.__set__, TypeError: an integer is required

Open
#40 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
17.3k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

Ubuntu 24.04.2 LTS, CUDA 12.4 Torch 2.5.1, have ffmpeg and
accelerate==1.6.0
av==14.0.1
gradio==5.25.2

error log as below:
=================
(video) tkadm@ubuntu2204:~/FramePack$ python demo_gradio_me.py

Currently enabled native sdp backends: ['flash', 'math', 'mem_efficient', 'cudnn']
Xformers is installed!
Flash Attn is installed!
Sage Attn is installed!
Namespace(share=False, server='0.0.0.0', port=10609)
Free VRAM 23.1300048828125 GB
High-VRAM Mode: False
Loading checkpoint shards: 100%|███████████████████████████████████████████| 4/4 [00:00<00:00, 4.77it/s]
Loading checkpoint shards: 100%|███████████████████████████████████████████| 3/3 [00:00<00:00, 13.33it/s]
transformer.high_quality_fp32_output_for_inference = True
* Running on local URL: http://0.0.0.0:10609

To create a public link, set `share=True` in `launch()`.
Unloaded DynamicSwap_LlamaModel as complete.
Unloaded CLIPTextModel as complete.
Unloaded SiglipVisionModel as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Unloaded DynamicSwap_HunyuanVideoTransformer3DModelPacked as complete.
Loaded CLIPTextModel to cuda:0 as complete.
Unloaded CLIPTextModel as complete.
Loaded AutoencoderKLHunyuanVideo to cuda:0 as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Loaded SiglipVisionModel to cuda:0 as complete.
latent_padding_size = 27, is_last_section = False
Unloaded SiglipVisionModel as complete.
Moving DynamicSwap_HunyuanVideoTransformer3DModelPacked to cuda:0 with preserved memory: 6 GB
100%|██████████████████████████████████████████████████| 25/25 [00:58<00:00, 2.34s/it]
Offloading DynamicSwap_HunyuanVideoTransformer3DModelPacked from cuda:0 to preserve memory: 8 GB
Loaded AutoencoderKLHunyuanVideo to cuda:0 as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.

Traceback (most recent call last):
File "/home/tkadm/FramePack/demo_gradio_me.py", line 294, in worker
save_bcthw_as_mp4(history_pixels, output_filename, fps=30)
File "/home/tkadm/FramePack/diffusers_helper/utils.py", line 279, in save_bcthw_as_mp4
torchvision.io.write_video(output_filename, x, fps=fps, video_codec='h264', options={'crf': '0'})
File "/home/tkadm/miniconda3/envs/video/lib/python3.11/site-packages/torchvision/io/video.py", line 140, in write_video
frame.pict_type = "NONE"
^^^^^^^^^^^^^^^
File "av/video/frame.pyx", line 204, in av.video.frame.VideoFrame.pict_type.__set__
TypeError: an integer is required
Unloaded DynamicSwap_LlamaModel as complete.
Unloaded CLIPTextModel as complete.
Unloaded SiglipVisionModel as complete.
Unloaded AutoencoderKLHunyuanVideo as complete.
Unloaded DynamicSwap_HunyuanVideoTransformer3DModelPacked as complete.

^CKeyboard interruption in main thread... closing server.

(video) tkadm@ubuntu2204:~/FramePack$ ffmpeg -version

ffmpeg version 7.0.2-static https://johnvansickle.com/ffmpeg/ Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 8 (Debian 8.3.0-6)
configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-libgme --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libdav1d --enable-libxvid --enable-libzvbi --enable-libzimg
libavutil 59. 8.100 / 59. 8.100
libavcodec 61. 3.100 / 61. 3.100
libavformat 61. 1.100 / 61. 1.100
libavdevice 61. 1.100 / 61. 1.100
libavfilter 10. 1.100 / 10. 1.100
libswscale 8. 1.100 / 8. 1.100
libswresample 5. 1.100 / 5. 1.100
libpostproc 58. 1.100 / 58. 1.100

I ckeck the file frame.pyx", line 204, as below:

@property
def pict_type(self):
"""Returns an integer that corresponds to the PictureType enum.

Wraps :ffmpeg:`AVFrame.pict_type`

:type: int
"""
return self.ptr.pict_type

I think it should be call ffmpeg to combine the image to mp4, but don't work.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in diffusers_helper/utils.py at save_bcthw_as_mp4 and inspect its torchvision.io.write_video call. Reproduce the failure with the listed Python package versions, especially av==14.0.1, and trace the pict_type assignment shown in the error. Done means the generated video is written without the TypeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
audio-video-rtc
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.