Latest PyAV version breaks torchvision.io.write_video when writing video with audio

Open
#6,814 6 comments 4 reactions 1 assignee View on GitHub

@bjuncek is already working on this.

Since Oct 24, 2022.

Assessment

This issue has not been assessed yet.

Description

dependency issue duplicate module: video
🐛 Describe the bug
Description

It seems the latest PyPi release of PyAV (av==10.0.0) breaks torchvision.io.write_video when writing a video containing audio. If you call write_video without audio array, it still seems to work fine - it's just when an audio array is provided that it breaks.

Reproducible Example

Here is a colab notebook reproducing the issue: Open In Colab
Here is that same notebook as GitHub gist: https://gist.github.com/nateraw/1123039fc90cefd90d282cf6c297d1d2

Strack Trace
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-3-28056a084160> in <module>
      6     audio_fps=44100,
      7     audio_codec='aac',
----> 8     audio_options=None
      9 )

/usr/local/lib/python3.7/dist-packages/torchvision/io/video.py in write_video(filename, video_array, fps, video_codec, options, audio_array, audio_fps, audio_codec, audio_options)
    114             num_channels = audio_array.shape[0]
    115             audio_layout = "stereo" if num_channels > 1 else "mono"
--> 116             audio_sample_fmt = container.streams.audio[0].format.name
    117 
    118             format_dtype = np.dtype(audio_format_dtypes[audio_sample_fmt])

IndexError: tuple index out of range
Temporary Solution

Downgrading to av==9.2.0 fixed the issue for me.

Versions
PyTorch version: 1.12.1+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
CMake version: version 3.22.6
Libc version: glibc-2.26

Python version: 3.7.15 (default, Oct 12 2022, 19:14:55)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic
Is CUDA available: False
CUDA runtime version: 11.2.152
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.1.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.1.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.6
[pip3] torch==1.12.1+cu113
[pip3] torchaudio==0.12.1+cu113
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.13.1
[pip3] torchvision==0.13.1+cu113
[conda] Could not collect
Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from pytorch/vision

All issues in pytorch/vision

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.