kwea123 / kwea123/nerf_pl

360 with own data not working

Open
#111 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.8k
Forks
458
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
I'm trying to prepare and train a model with my own dataset of images but I'm having some troubles for 360º.
On the one hand, there are some cases where, using the collab code of COLMAP, it appears an error about not finding the poses of the camera. Is this because of the images are wrong?

On the other hand, with the dataset and COLMAP executed correctly, I train the model with that files but, when I run the "eval.py" to check it, the images are not of the object, just some black or white images with some noise. Here I share some images used for the training.

![frame14](https://user-images.githubusercontent.com/69549100/143689123-fff921fb-314b-401f-aefc-db40a68a48ed.png)
![frame112](https://user-images.githubusercontent.com/69549100/143689233-35fb504d-a42b-46a5-b3ed-f5dcd67c24ef.png)
![frame224](https://user-images.githubusercontent.com/69549100/143689226-a349c109-ca13-48dd-a9b2-6e1382195765.png)

And the resulting images are all like these:
![008](https://user-images.githubusercontent.com/69549100/143689170-5e2232c3-cc2c-46df-9820-902c0874cd45.png)
![042](https://user-images.githubusercontent.com/69549100/143689171-fdd1b2f3-71e9-41ca-b0ad-7654ecbbd55f.png)
![084](https://user-images.githubusercontent.com/69549100/143689172-123e52c6-09a3-41d4-bb9a-1d5d96176394.png)
![000](https://user-images.githubusercontent.com/69549100/143689173-631125f2-b393-44de-8d78-e0b8ff585891.png)

Finally, I don't know where the problem is so I would like to know if it's just that the --spheric is not working well or some problems with the original images.

**Which branch you use**
I'm currently working with the dev branch.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start on the dev branch by reproducing the COLMAP pose-generation step with the supplied image setup, then run eval.py using the generated files. Compare the pose-generation result with the black or noisy evaluation images to determine whether the failure is in the input images, camera poses, or --spheric handling; done means the failing stage and reproducible cause are identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python, pytorch
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.