hzwer / hzwer/Practical-RIFE

torch.cat ERROR

Open
#16 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1k
Forks
131
PR merge metrics
No merged PRs in 30d

Description

I'm using 4.1 model. When I try to interpolarate a video, I get such a ERROR:

2022-07-24 19:29:27,296 - one_line_shot_args - 772 - INFO - Initial New Interpolation Project: project_dir: C:/Users/Globefish/Downloads/Compressed/SVFI.3.2.Community\\u3010MAD\u3011 Genuine article \u3010\u4ffa\u30ac\u30a4\u30eb\u3011_1, INPUT_FILEPATH: C:/Users/Globefish/Desktop/\u3010MAD\u3011 Genuine article \u3010\u4ffa\u30ac\u30a4\u30eb\u3011_1.mp4
2022-07-24 19:29:27,296 - one_line_shot_args - 774 - INFO - Changing working dir to C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package
2022-07-24 19:29:27,296 - one_line_shot_args - 781 - WARNING - Not find selected ffmpeg, use default
2022-07-24 19:29:27,376 - one_line_shot_args - 255 - INFO - 
Input Video Info
{'index': 0, 'width': 960, 'height': 540, 'color_range': 'tv', 'color_space': 'smpte170m', 'color_transfer': 'smpte170m', 'color_primaries': 'smpte170m', 'r_frame_rate': '30/1', 'duration': '230.500000', 'nb_frames': '6915', 'disposition': {'default': 1, 'dub': 0, 'original': 0, 'comment': 0, 'lyrics': 0, 'karaoke': 0, 'forced': 0, 'hearing_impaired': 0, 'visual_impaired': 0, 'clean_effects': 0, 'attached_pic': 0, 'timed_thumbnails': 0}, 'tags': {'creation_time': '2022-07-24T04:25:41.000000Z', 'language': 'eng', 'handler_name': '\x1fMainconcept Video Media Handler', 'encoder': 'AVC Coding'}}
2022-07-24 19:29:27,376 - one_line_shot_args - 273 - INFO - Auto Find FPS in r_frame_rate: 30.0
2022-07-24 19:29:27,376 - one_line_shot_args - 280 - INFO - Auto Find frames cnt in nb_frames: 6915
2022-07-24 19:29:27,390 - one_line_shot_args - 832 - INFO - Check Interpolation Source, FPS: 30.0, TARGET FPS: 60.0, FRAMES_CNT: 13830, EXP: 1
2022-07-24 19:29:27,407 - one_line_shot_args - 859 - INFO - Buffer Size to 677
2022-07-24 19:29:29,077 - one_line_shot_args - 1412 - INFO - Start VRAM Test: 960x540 with scale 1.0
2022-07-24 19:29:30,384 - one_line_shot_args - 1420 - ERROR - VRAM Check Failed, PLS Lower your presets
Traceback (most recent call last):
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\one_line_shot_args.py", line 1416, in nvidia_vram_test
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\Utils\inference.py", line 174, in generate_interp
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\Utils\inference.py", line 87, in __make_inference
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\model\RIFE_HDv3.py", line 60, in inference
flow, mask, merged = self.flownet(imgs, timestep, scale_list)
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\model\IFNet_HDv3.py", line 99, in forward
f0, m0 = block[i](torch.cat((warped_img0[:, :3], warped_img1[:, :3], timestep, mask), 1), flow, scale=scale_list[i])
RuntimeError: Sizes of tensors must match except in dimension 3. Got 576 and 544 (The offending index is 2)

Traceback (most recent call last):
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\one_line_shot_args.py", line 2151, in <module>
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\one_line_shot_args.py", line 1871, in run
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\one_line_shot_args.py", line 1416, in nvidia_vram_test
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\Utils\inference.py", line 174, in generate_interp
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\Utils\inference.py", line 87, in __make_inference
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\model\RIFE_HDv3.py", line 60, in inference
flow, mask, merged = self.flownet(imgs, timestep, scale_list)
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\torch\nn\modules\module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "C:\Users\GLOBEF~1\DOWNLO~1\COMPRE~1\SVFI32~1.COM\Package\model\IFNet_HDv3.py", line 99, in forward
f0, m0 = block[i](torch.cat((warped_img0[:, :3], warped_img1[:, :3], timestep, mask), 1), flow, scale=scale_list[i])
RuntimeError: Sizes of tensors must match except in dimension 3. Got 576 and 544 (The offending index is 2)
INFO - ONE LINE SHOT ARGS 6.6.1 2021/6/26
Warning: Find Empty Args at 'ffmpeg_customized'
Warning: Find Empty Args at 'slow_motion_fps'
Warning: Find Empty Args at 'resize'
Warning: Find Empty Args at 'crop'
Warning: Find Empty Args at 'use_sr_model'
INFO - FP16 mode switch success
INFO - Loaded v3.x HD model.

If i use a "scale_factor" of 0.5 (though the original video is only 960x540), it can work well.
I'm using a RTX3060 Laptop with 6GB VRAM, I guess VRAM is enough. The same video can be use as input for 3.9 model with 1.0 scale factor.

I wonder how to solve this issue, thanks.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at model/IFNet_HDv3.py line 99, where torch.cat raises the dimension mismatch, and inspect Utils/inference.py plus nvidia_vram_test in one_line_shot_args.py. Reproduce with the reported 960x540 video at scale 1.0, compare the tensor dimensions reaching the concatenation, and verify interpolation completes without the error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.