facebookresearch / facebookresearch/hyperreel
training my own video dataset
- Dominant language
- Python
- Stars
- 481
- Forks
- 33
- PR merge metrics
- No merged PRs in 30d
Description
I am trying to train my own inward facing video dataset with 30 cameras. I am unable to understand the coordinate system of the code in hyperreel. My current poses are all in openCV coordinate system x right, y down and z forward notation. How can I translate of use my transform matrix with your code?
I trying building on this using immersive.py code base and immersive_sphere.yml. But my model doesn't seem to learn the volume. I am getting bllurred/ colourful novel views which is really weird. Do you have any suggestion for this? Please tell me a way to use my openCV coordinate system based transform and intrinsics and train the model using your code! Also,what is the most ideal model for my case with inward facing scene/ video data/ openCV coordinates?
Please respond.
Contributor guide
Research direction
Start by reading immersive.py and immersive_sphere.yml, then trace how the training code represents poses and intrinsics relative to the stated OpenCV coordinate system. Reproduce the training setup and determine whether the coordinate conversion and inward-facing model configuration are documented or supported; done means the issue has a verified path for training this dataset without blurred or colourful novel views.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100