facebookresearch / facebookresearch/hyperreel

training my own video dataset

Open
#20 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
481
Forks
33
PR merge metrics
No merged PRs in 30d

Description

I am trying to train my own inward facing video dataset with 30 cameras. I am unable to understand the coordinate system of the code in hyperreel. My current poses are all in openCV coordinate system x right, y down and z forward notation. How can I translate of use my transform matrix with your code?
I trying building on this using immersive.py code base and immersive_sphere.yml. But my model doesn't seem to learn the volume. I am getting bllurred/ colourful novel views which is really weird. Do you have any suggestion for this? Please tell me a way to use my openCV coordinate system based transform and intrinsics and train the model using your code! Also,what is the most ideal model for my case with inward facing scene/ video data/ openCV coordinates?
Please respond.

Contributor guide

Open the contributing guide

Research direction

Start by reading immersive.py and immersive_sphere.yml, then trace how the training code represents poses and intrinsics relative to the stated OpenCV coordinate system. Reproduce the training setup and determine whether the coordinate conversion and inward-facing model configuration are documented or supported; done means the issue has a verified path for training this dataset without blurred or colourful novel views.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.