ByteDance-Seed / ByteDance-Seed/SeedVR

Slow Inference Speed and Low GPU Utilization with SeedVR2-7B on H100

Open
#15 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.4k
Forks
82
PR merge metrics
No merged PRs in 30d

Description

Hi team,

First of all, thank you for your incredible work on this project. It's truly a great contribution.

I'm currently using SeedVR2-7B for image processing and have encountered a performance issue I'd like to ask about.

### Environment
* **Model:** SeedVR2-7B
* **Hardware:** 4 x NVIDIA H100 80G
* **Input Data:** Images of size 720x480.
* **Script:** `inference_seedvr2_7b.py`

### The Issue

My initial goal was to process a video file directly. The video was 30 seconds long with a resolution of 720x480. I used four H100 80G GPUs and set the `--sp_size=4` flag. However, this resulted in an Out-of-Memory (OOM) error.

As a workaround, I extracted all frames from the video and started processing them as individual images. The image resolution is also 720x480. This approach successfully avoided the OOM error on a single H100.

However, the inference speed is extremely slow. On a single H100 80G GPU, it takes approximately **18 hours to process around 1200 images**.

To provide more context on how I'm running the inference, I want to clarify that I'm running **independent inference processes on each GPU**, not a single multi-GPU process.

Here is the exact command I use to launch a process on a single, specific GPU (e.g., GPU 3):

```bash
CUDA_VISIBLE_DEVICES=3 PYTHONPATH=$PWD torchrun --nproc-per-node=1 --master_port=29511 \
projects/inference_seedvr2_7b_1.py \
--video_path [MY_DIR/frames_29.97_split_1] \
--output_dir [MY_DIR/output_split_1]
```

**Note:**
* I run multiple instances of this command in parallel, each with a different `CUDA_VISIBLE_DEVICES` value (e.g., 0, 1, 2, 3) and a different `master_port`.
* The `--video_path` argument points to a directory containing the pre-extracted image frames for that split.

### GPU Utilization Observation

I monitored the GPU performance while the script was running. The GPU was dedicated solely to this task.

I observed that while the GPU memory usage is consistently high (as expected, even for single images), the GPU compute utilization is very low for most of the time, with only occasional spikes. This suggests that the GPU's computational power is not being fully leveraged.

*(Please see the attached screenshot of the GPU monitoring tool below)*

Image

### My Questions

1. Is this low GPU compute utilization an inherent characteristic of the model's architecture, or could it be a bottleneck in the data loading/preprocessing pipeline that can be optimized?
2. Are there any recommended methods, configurations, or optimizations to improve the inference speed for processing individual images?

### Minor Bug Report

On a side note, I noticed that the `inference_seedvr2_7b.py` script does not seem to save the output images correctly when processing a single image. I had to make a small modification to the script to get it to work as expected.

Thank you for your time and any insights you can provide. I'm looking forward to your feedback.

Best regards.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.