openframeworks / openframeworks/openFrameworks

ofVideoPlayer: vertical videos showing up rotated -90 (video metadata not being read)

Open
#5,501 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature macOS proposed-solution section-video
Dominant language
C++
Stars
10.4k
Forks
2.6k
Avg merge
1d 21h
Merged PRs (30d)
9

Description

I was trying to use a vertical video (1080w x 1920h) shot on my mobile (android) with ofVideoPlayer, but as I said in the title, the video is drawn rotated -90 counter-clockwise. I'm using El Capitan and oF 0.9.8. Is this a known issue or intended behavior? I did some searching but didn't find any other issues or posts about this. I'm just doing

// in ofApp.h
ofVideoPlayer video;

// in setup()
video.load("video.mp4"); 
video.play();

// in update()
video.update();

// in draw()
video.draw(0,0);

I took a peek inside the video metadata and found this.
screen shot 2017-03-20 at 1 02 12 pm

So it has two relevant looking fields

Metadata:
    rotate : 90

And

Side data: 
    displaymatrix: rotation of -90.00 degrees

So I can think either one of two things is happening.

  1. oF is seeing this displaymatrix info and rotating the video -90.
  2. the video is natively at -90 and oF is not seeing the metadata (even though OSX finder-preview and quicktime display correctly).

Re-encoding the video (ffmpeg -i video.mp4 out.mp4) fixed the issue, likely because it dumped all the original metadata.

FWIW I have also noticed this problem when building for iOS and drawing videos pulled from the phone's media library using the AVFoundationVideoPlayer.

Messing around in the ofAVFoundationVideoPlayer.m I was able to get the video orientation with the bit below, but preferredTransform is readOnly in AssetTrack. So it looks like you would have to do something else (possibly with AVMutableCompositionTrack?) to set the correct orientation.

CGAffineTransform tf       = [videoTrack preferredTransform];
CGFloat videoAngleInDegree  = (atan2(tf.b, tf.a));
videoAngleInDegree *= 180 / M_PI;
NSLog(@"degrees = %f", videoAngleInDegree);

I uploaded the video here if anyone wants to test with it.
https://www.dropbox.com/s/l63161hgq3utcmy/dume.mp4?dl=0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the rotation problem with the supplied dume.mp4 and the ofVideoPlayer example, then inspect ofAVFoundationVideoPlayer.m and the platform video-player paths mentioned in the issue. Compare the video metadata and AVFoundation preferredTransform behavior on macOS and iOS. Done means the supplied vertically recorded video draws upright without requiring re-encoding on the affected platforms.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, ios, macos, objective-c
Domain
computer-graphics, desktop-dev, mobile-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.